2024-04-25T12:47:33.984 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T12:47:33.985 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} -- /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/orig.config.yaml 2024-04-25T12:47:34.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T12:47:34.183 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} email: null first_in_suite: false job_id: '7673453' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync rbytes: false syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4060 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7sPohrFa/ycNsFwzLrFpz+tFbDl/kKeJBi5ZXHCpl8jNxfniMPBXsVBzQP5RTkXPPkSpwM+KCQOPp5JB3mGGY= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJIK5jD5ANCb/OquUGo45AIch0aciP7DNLp7PDAlntG2ktHGtcIWL0sqo8MAFIvr1iyUOOxIAu25mqnUgfReBSw= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxexWztidPuKK87DQUoYQxOHZKXtWTDMt67g8lV0g6JLZVF9NUfIrNeqpayPj/Otx9W8DsVWqHLXOqWmaZmWvQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_12:17:05 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T12:47:34.183 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa; will attempt to use it 2024-04-25T12:47:34.184 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks 2024-04-25T12:47:34.184 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T12:47:34.185 INFO:teuthology.task.internal:Checking packages... 2024-04-25T12:47:34.209 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T12:47:34.209 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T12:47:34.209 INFO:teuthology.packaging:ref: None 2024-04-25T12:47:34.209 INFO:teuthology.packaging:tag: None 2024-04-25T12:47:34.209 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T12:47:34.209 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:47:34.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T12:47:34.448 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T12:47:34.449 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T12:47:34.456 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T12:47:34.456 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T12:47:34.495 INFO:teuthology.task.internal:Saving configuration 2024-04-25T12:47:34.516 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T12:47:34.523 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:47:34.547 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:42:10.113651', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7sPohrFa/ycNsFwzLrFpz+tFbDl/kKeJBi5ZXHCpl8jNxfniMPBXsVBzQP5RTkXPPkSpwM+KCQOPp5JB3mGGY='} 2024-04-25T12:47:34.569 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:42:10.112037', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJIK5jD5ANCb/OquUGo45AIch0aciP7DNLp7PDAlntG2ktHGtcIWL0sqo8MAFIvr1iyUOOxIAu25mqnUgfReBSw='} 2024-04-25T12:47:34.590 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:42:10.110136', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxexWztidPuKK87DQUoYQxOHZKXtWTDMt67g8lV0g6JLZVF9NUfIrNeqpayPj/Otx9W8DsVWqHLXOqWmaZmWvQ='} 2024-04-25T12:47:34.590 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T12:47:34.597 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T12:47:34.598 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T12:47:34.598 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T12:47:34.598 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T12:47:34.717 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f442a3490d0>, signals=[15]) 2024-04-25T12:47:34.718 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T12:47:34.725 INFO:teuthology.task.internal:Opening connections... 2024-04-25T12:47:34.725 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T12:47:34.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:47:34.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-04-25T12:47:34.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:47:34.879 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T12:47:34.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:47:34.952 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T12:47:34.960 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-04-25T12:47:34.979 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-04-25T12:47:34.979 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-04-25T12:47:35.037 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-04-25T12:47:35.037 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-04-25T12:47:35.037 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-04-25T12:47:35.037 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:47:35.037 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:47:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:47:35.039 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-04-25T12:47:35.074 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-04-25T12:47:35.091 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-04-25T12:47:35.091 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-04-25T12:47:35.148 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-04-25T12:47:35.148 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-04-25T12:47:35.148 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-04-25T12:47:35.148 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:47:35.148 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-04-25T12:47:35.148 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:47:35.149 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:47:35.149 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-04-25T12:47:35.174 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-04-25T12:47:35.191 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-04-25T12:47:35.192 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-04-25T12:47:35.249 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-04-25T12:47:35.249 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-04-25T12:47:35.249 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-04-25T12:47:35.249 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:47:35.249 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:47:35.250 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:47:35.250 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-04-25T12:47:35.274 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T12:47:35.285 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T12:47:35.350 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T12:47:35.350 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:47:35.353 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:47:35.355 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:47:35.371 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T12:47:35.379 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T12:47:35.379 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:47:35.411 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:47:35.413 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:47:35.438 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T12:47:35.516 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T12:47:35.531 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T12:47:35.531 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T12:47:35.531 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T12:47:35.532 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:47:35.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:47:35.549 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-04-25T12:47:35.607 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-437.el9.x86_64 2024-04-25T12:47:35.607 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-437.el9.x86_64 2024-04-25T12:47:35.607 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-04-25T12:47:38.038 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T12:47:40.256 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T12:47:44.731 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.2 MB 00:00 2024-04-25T12:47:46.305 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-25T12:47:48.317 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T12:47:53.556 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-25T12:47:55.546 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T12:47:55.546 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T12:47:55.629 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T12:47:55.630 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-04-25T12:47:55.630 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-25T12:47:55.738 DEBUG:teuthology.orchestra.run.smithi038:> echo no | sudo yum reinstall kernel || true 2024-04-25T12:47:56.259 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:47:53 PM UTC. 2024-04-25T12:47:56.382 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T12:47:56.393 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T12:47:56.394 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-04-25T12:47:56.394 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T12:47:56.394 INFO:teuthology.orchestra.run.smithi038.stdout:Reinstalling: 2024-04-25T12:47:56.394 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T12:47:56.394 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:47:56.395 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-25T12:47:56.395 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T12:47:56.395 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:47:56.395 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 255 k 2024-04-25T12:47:56.395 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 0 2024-04-25T12:47:56.396 INFO:teuthology.orchestra.run.smithi038.stderr:Operation aborted. 2024-04-25T12:47:56.454 INFO:teuthology.orchestra.run.smithi038.stdout:Is this ok [y/N]: 2024-04-25T12:47:56.454 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum reinstall -y kernel || true 2024-04-25T12:47:56.971 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:47:53 PM UTC. 2024-04-25T12:47:57.092 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T12:47:57.093 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T12:47:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-04-25T12:47:57.093 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T12:47:57.093 INFO:teuthology.orchestra.run.smithi038.stdout:Reinstalling: 2024-04-25T12:47:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 255 k 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 0 2024-04-25T12:47:57.094 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-04-25T12:47:57.666 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-437.el9.x86_64.rpm 728 kB/s | 255 kB 00:00 2024-04-25T12:47:57.667 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:47:57.667 INFO:teuthology.orchestra.run.smithi038.stdout:Total 444 kB/s | 255 kB 00:00 2024-04-25T12:47:57.688 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-04-25T12:47:57.878 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-04-25T12:47:57.878 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-04-25T12:47:57.883 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-04-25T12:47:57.883 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-04-25T12:47:58.007 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-04-25T12:47:58.101 INFO:teuthology.orchestra.run.smithi038.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T12:47:58.102 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:47:58.900 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:47:58.900 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T12:47:59.573 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T12:47:59.573 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:47:59.573 INFO:teuthology.orchestra.run.smithi038.stdout:Reinstalled: 2024-04-25T12:47:59.573 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T12:47:59.573 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:47:59.573 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-25T12:47:59.747 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T12:47:59.797 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T12:47:59.798 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T12:47:59.798 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T12:47:59.798 DEBUG:teuthology.parallel:result is None 2024-04-25T12:47:59.798 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T12:47:59.808 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T12:47:59.808 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:47:59.810 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:47:59.813 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:47:59.830 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T12:47:59.869 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T12:47:59.895 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T12:47:59.895 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:47:59.898 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:47:59.900 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:48:00.179 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T12:48:00.192 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T12:48:00.192 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:48:00.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:48:00.208 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:48:00.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:48:00.225 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:48:00.227 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:48:00.251 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:48:00.269 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:48:00.295 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:48:00.303 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:48:00.304 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:48:00.308 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:48:00.334 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:48:00.336 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T12:48:00.344 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T12:48:00.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:48:00.347 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:48:00.350 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:48:00.402 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T12:48:00.413 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T12:48:00.413 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:48:00.416 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:48:00.418 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:48:00.457 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:48:00.509 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:48:00.560 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:48:00.641 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:48:00.683 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:48:00.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:48:00.749 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:48:00.788 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:48:00.843 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:48:00.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:48:00.968 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:48:00.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:48:01.034 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:48:01.078 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:48:01.123 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:48:01.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:48:01.250 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:48:01.250 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:48:01.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-04-25T12:48:01.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-04-25T12:48:01.321 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-04-25T12:48:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:48:01.354 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:48:01.396 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:48:01.857 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T12:48:01.865 INFO:teuthology.task.internal:Starting timer... 2024-04-25T12:48:01.865 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T12:48:01.905 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T12:48:01.953 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T12:48:01.954 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-04-25T12:48:02.010 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-04-25T12:48:02.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-04-25T12:48:02.085 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-04-25T12:48:02.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-04-25T12:48:02.151 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-04-25T12:48:02.153 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T12:48:02.153 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-04-25T12:48:02.176 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-04-25T12:48:02.176 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-04-25T12:48:02.210 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-04-25T12:48:02.210 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-04-25T12:48:02.231 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-04-25T12:48:02.232 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-04-25T12:48:02.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:48:02.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:48:02.258 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:48:02.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:48:02.285 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:48:02.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:48:02.311 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T12:48:02.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:48:02.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:48:02.360 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:48:02.385 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T12:48:02.396 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T12:48:02.401 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T12:48:02.571 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T12:48:02.587 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T12:48:02.589 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T12:51:47.392 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-04-25T12:51:47.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-04-25T12:51:47.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:51:47.471 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-04-25T12:51:47.557 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-04-25T12:51:47.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-25T12:51:47.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:51:47.630 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-04-25T12:51:47.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-25T12:51:47.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-25T12:51:47.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:51:47.791 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-04-25T12:51:47.879 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-25T12:51:47.879 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T12:51:47.890 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T12:51:47.891 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:51:47.891 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:51:47.893 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:51:47.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:51:47.896 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:51:47.896 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:51:47.921 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:51:47.923 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:51:47.934 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:51:47.937 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:51:47.951 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:51:47.956 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-04-25T12:51:47.961 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-04-25T12:51:47.965 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:51:47.966 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-04-25T12:51:47.972 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-04-25T12:51:47.979 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:51:47.987 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:51:47.989 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-04-25T12:51:47.992 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:51:48.000 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-04-25T12:51:48.000 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:51:48.013 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:51:48.026 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:51:48.056 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:51:48.059 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:51:48.059 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-04-25T12:51:48.060 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.060 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.060 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.060 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.060 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:51:48.065 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:51:48.065 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-04-25T12:51:48.065 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.065 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.065 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.065 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.085 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:51:48.089 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:51:48.089 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-04-25T12:51:48.089 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.089 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.089 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.089 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:51:48.090 INFO:teuthology.run_tasks:Running task install... 2024-04-25T12:51:48.097 DEBUG:teuthology.task.install:project ceph 2024-04-25T12:51:48.097 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T12:51:48.097 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T12:51:48.097 INFO:teuthology.task.install:Using flavor: default 2024-04-25T12:51:48.103 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T12:51:48.103 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T12:51:48.103 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:51:48.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:51:48.106 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:51:48.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:51:48.107 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T12:51:48.107 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:51:48.329 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:51:48.330 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:51:48.333 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:51:48.333 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:51:48.336 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T12:51:48.337 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T12:51:48.603 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:51:48.603 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:51:48.603 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:51:48.628 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:51:48.628 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:51:48.641 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:51:48.641 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:51:48.641 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:51:48.670 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:51:48.670 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:51:48.693 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:51:48.739 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:51:48.766 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:51:48.817 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:51:48.829 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-04-25T12:51:48.831 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-04-25T12:51:48.883 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-04-25T12:51:48.884 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-04-25T12:51:49.055 INFO:teuthology.orchestra.run.smithi178.stdout:48 files removed 2024-04-25T12:51:49.082 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:51:49.121 INFO:teuthology.orchestra.run.smithi038.stdout:48 files removed 2024-04-25T12:51:49.148 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:51:49.850 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 194 kB/s | 83 kB 00:00 2024-04-25T12:51:49.862 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 160 kB/s | 83 kB 00:00 2024-04-25T12:51:50.296 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T12:51:50.368 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-25T12:51:50.805 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-25T12:51:51.068 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-04-25T12:51:51.474 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:51:51.474 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:51:51.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:51:51.475 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T12:51:51.504 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:51:51.505 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:51:51.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:51:51.648 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:51:51.714 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-04-25T12:51:51.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-04-25T12:51:51.941 INFO:teuthology.orchestra.run.smithi104.stdout:48 files removed 2024-04-25T12:51:51.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:51:52.686 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-04-25T12:51:53.169 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-25T12:51:53.513 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-04-25T12:51:53.695 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-25T12:51:53.933 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T12:51:54.548 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-04-25T12:51:55.522 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T12:51:56.424 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T12:51:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T12:51:59.666 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T12:52:00.073 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-04-25T12:52:00.516 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T12:52:01.311 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T12:52:01.955 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-25T12:52:02.529 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T12:52:03.459 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T12:52:04.456 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T12:52:06.450 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T12:52:08.506 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T12:52:08.618 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:52:08.619 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:52:08.629 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:52:08.630 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:52:08.631 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:52:08.632 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:52:08.633 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:52:08.636 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:52:08.636 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:52:08.697 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-25T12:52:08.714 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-04-25T12:52:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-04-25T12:52:08.714 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-04-25T12:52:08.714 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-04-25T12:52:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:52:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:52:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:52:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:52:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:52:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:52:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:52:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:52:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:52:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:52:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:52:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:52:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout:Install 148 Packages 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 4 Packages 2024-04-25T12:52:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:52:08.727 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 222 M 2024-04-25T12:52:08.727 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-04-25T12:52:09.839 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T12:52:10.640 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:52:10.641 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:52:10.652 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:52:10.653 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:52:10.654 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:52:10.655 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:52:10.656 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:52:10.659 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:52:10.660 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:52:10.721 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-25T12:52:10.726 INFO:teuthology.orchestra.run.smithi178.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:52:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:52:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:52:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:52:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:52:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:52:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:52:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:52:10.752 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:52:10.753 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:52:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:52:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout:Install 148 Packages 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 4 Packages 2024-04-25T12:52:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:52:10.757 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 222 M 2024-04-25T12:52:10.757 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-25T12:52:11.168 INFO:teuthology.orchestra.run.smithi178.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 890 kB 00:00 2024-04-25T12:52:11.268 INFO:teuthology.orchestra.run.smithi178.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-25T12:52:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 5.7 MB/s | 5.4 MB 00:00 2024-04-25T12:52:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 1.5 MB 00:00 2024-04-25T12:52:11.699 INFO:teuthology.orchestra.run.smithi104.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-25T12:52:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 9.6 MB/s | 4.7 MB 00:00 2024-04-25T12:52:12.119 INFO:teuthology.orchestra.run.smithi178.stdout:(7/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-25T12:52:12.191 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:52:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:52:12.203 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:52:12.203 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:52:12.204 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:52:12.205 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:52:12.207 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:52:12.209 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:52:12.210 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:52:12.273 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T12:52:12.283 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T12:52:12.284 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-04-25T12:52:12.285 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T12:52:12.286 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:52:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:52:12.288 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:52:12.289 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:52:12.290 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:52:12.291 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:52:12.292 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-25T12:52:12.293 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-25T12:52:12.294 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-04-25T12:52:12.294 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 4 Packages 2024-04-25T12:52:12.294 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:52:12.294 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 222 M 2024-04-25T12:52:12.294 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-04-25T12:52:12.449 INFO:teuthology.orchestra.run.smithi104.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 890 kB 00:00 2024-04-25T12:52:12.515 INFO:teuthology.orchestra.run.smithi104.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-25T12:52:12.603 INFO:teuthology.orchestra.run.smithi178.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 9.6 MB/s | 21 MB 00:02 2024-04-25T12:52:12.670 INFO:teuthology.orchestra.run.smithi178.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 379 kB/s | 25 kB 00:00 2024-04-25T12:52:13.208 INFO:teuthology.orchestra.run.smithi104.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 2.4 MB 00:00 2024-04-25T12:52:13.333 INFO:teuthology.orchestra.run.smithi038.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-25T12:52:13.616 INFO:teuthology.orchestra.run.smithi104.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-25T12:52:13.686 INFO:teuthology.orchestra.run.smithi178.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 9.9 MB/s | 17 MB 00:01 2024-04-25T12:52:13.786 INFO:teuthology.orchestra.run.smithi178.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-25T12:52:13.937 INFO:teuthology.orchestra.run.smithi178.stdout:(12/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 5.8 MB/s | 11 MB 00:01 2024-04-25T12:52:13.958 INFO:teuthology.orchestra.run.smithi038.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T12:52:13.962 INFO:teuthology.orchestra.run.smithi178.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 154 kB/s | 27 kB 00:00 2024-04-25T12:52:14.020 INFO:teuthology.orchestra.run.smithi178.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-25T12:52:14.091 INFO:teuthology.orchestra.run.smithi104.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-25T12:52:14.092 INFO:teuthology.orchestra.run.smithi038.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-04-25T12:52:14.113 INFO:teuthology.orchestra.run.smithi178.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.9 MB/s | 747 kB 00:00 2024-04-25T12:52:14.138 INFO:teuthology.orchestra.run.smithi178.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-25T12:52:14.205 INFO:teuthology.orchestra.run.smithi178.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.3 MB/s | 126 kB 00:00 2024-04-25T12:52:14.255 INFO:teuthology.orchestra.run.smithi178.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-25T12:52:14.283 INFO:teuthology.orchestra.run.smithi104.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 7.0 MB/s | 4.7 MB 00:00 2024-04-25T12:52:14.338 INFO:teuthology.orchestra.run.smithi178.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 541 kB/s | 45 kB 00:00 2024-04-25T12:52:14.464 INFO:teuthology.orchestra.run.smithi178.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.1 MB/s | 135 kB 00:00 2024-04-25T12:52:14.589 INFO:teuthology.orchestra.run.smithi178.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.3 MB/s | 161 kB 00:00 2024-04-25T12:52:14.698 INFO:teuthology.orchestra.run.smithi178.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 320 kB 00:00 2024-04-25T12:52:14.815 INFO:teuthology.orchestra.run.smithi178.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 298 kB 00:00 2024-04-25T12:52:14.934 INFO:teuthology.orchestra.run.smithi178.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 834 kB/s | 99 kB 00:00 2024-04-25T12:52:15.043 INFO:teuthology.orchestra.run.smithi178.stdout:(25/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.4 MB/s | 5.4 MB 00:00 2024-04-25T12:52:15.068 INFO:teuthology.orchestra.run.smithi178.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 637 kB/s | 85 kB 00:00 2024-04-25T12:52:15.177 INFO:teuthology.orchestra.run.smithi178.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-25T12:52:15.293 INFO:teuthology.orchestra.run.smithi178.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 293 kB/s | 34 kB 00:00 2024-04-25T12:52:15.419 INFO:teuthology.orchestra.run.smithi178.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.1 MB/s | 142 kB 00:00 2024-04-25T12:52:15.458 INFO:teuthology.orchestra.run.smithi038.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-25T12:52:15.819 INFO:teuthology.orchestra.run.smithi178.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.0 MB/s | 3.1 MB 00:00 2024-04-25T12:52:16.225 INFO:teuthology.orchestra.run.smithi038.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 1.5 MB 00:00 2024-04-25T12:52:16.294 INFO:teuthology.orchestra.run.smithi178.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.0 MB/s | 3.5 MB 00:00 2024-04-25T12:52:16.366 INFO:teuthology.orchestra.run.smithi104.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 5.1 MB/s | 11 MB 00:02 2024-04-25T12:52:16.394 INFO:teuthology.orchestra.run.smithi178.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-25T12:52:16.441 INFO:teuthology.orchestra.run.smithi104.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 336 kB/s | 25 kB 00:00 2024-04-25T12:52:16.495 INFO:teuthology.orchestra.run.smithi178.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 493 kB/s | 49 kB 00:00 2024-04-25T12:52:16.604 INFO:teuthology.orchestra.run.smithi178.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T12:52:16.720 INFO:teuthology.orchestra.run.smithi178.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 273 kB 00:00 2024-04-25T12:52:16.921 INFO:teuthology.orchestra.run.smithi178.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.7 MB/s | 762 kB 00:00 2024-04-25T12:52:17.000 INFO:teuthology.orchestra.run.smithi104.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 5.8 MB/s | 17 MB 00:02 2024-04-25T12:52:17.012 INFO:teuthology.orchestra.run.smithi178.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T12:52:17.084 INFO:teuthology.orchestra.run.smithi104.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.6 MB/s | 225 kB 00:00 2024-04-25T12:52:17.113 INFO:teuthology.orchestra.run.smithi178.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 806 kB/s | 80 kB 00:00 2024-04-25T12:52:17.184 INFO:teuthology.orchestra.run.smithi104.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-25T12:52:17.188 INFO:teuthology.orchestra.run.smithi178.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2024-04-25T12:52:17.230 INFO:teuthology.orchestra.run.smithi178.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T12:52:17.259 INFO:teuthology.orchestra.run.smithi104.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-25T12:52:17.272 INFO:teuthology.orchestra.run.smithi178.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T12:52:17.300 INFO:teuthology.orchestra.run.smithi038.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-25T12:52:17.330 INFO:teuthology.orchestra.run.smithi178.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-04-25T12:52:17.364 INFO:teuthology.orchestra.run.smithi178.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T12:52:17.398 INFO:teuthology.orchestra.run.smithi178.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T12:52:17.417 INFO:teuthology.orchestra.run.smithi104.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 747 kB 00:00 2024-04-25T12:52:17.433 INFO:teuthology.orchestra.run.smithi178.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.6 MB/s | 125 kB 00:00 2024-04-25T12:52:17.501 INFO:teuthology.orchestra.run.smithi104.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 163 kB 00:00 2024-04-25T12:52:17.523 INFO:teuthology.orchestra.run.smithi178.stdout:(46/152): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-04-25T12:52:17.549 INFO:teuthology.orchestra.run.smithi178.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-25T12:52:17.568 INFO:teuthology.orchestra.run.smithi104.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-25T12:52:17.574 INFO:teuthology.orchestra.run.smithi178.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T12:52:17.660 INFO:teuthology.orchestra.run.smithi104.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-25T12:52:17.676 INFO:teuthology.orchestra.run.smithi178.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 943 kB 00:00 2024-04-25T12:52:17.783 INFO:teuthology.orchestra.run.smithi178.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 516 kB/s | 107 kB 00:00 2024-04-25T12:52:17.808 INFO:teuthology.orchestra.run.smithi178.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 163 kB/s | 21 kB 00:00 2024-04-25T12:52:17.817 INFO:teuthology.orchestra.run.smithi038.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.0 MB/s | 4.7 MB 00:01 2024-04-25T12:52:17.833 INFO:teuthology.orchestra.run.smithi178.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T12:52:17.859 INFO:teuthology.orchestra.run.smithi178.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-04-25T12:52:17.926 INFO:teuthology.orchestra.run.smithi178.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 32 MB/s | 3.0 MB 00:00 2024-04-25T12:52:17.951 INFO:teuthology.orchestra.run.smithi178.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 204 kB/s | 18 kB 00:00 2024-04-25T12:52:17.976 INFO:teuthology.orchestra.run.smithi178.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-04-25T12:52:18.002 INFO:teuthology.orchestra.run.smithi178.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T12:52:18.027 INFO:teuthology.orchestra.run.smithi178.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-25T12:52:18.052 INFO:teuthology.orchestra.run.smithi178.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T12:52:18.077 INFO:teuthology.orchestra.run.smithi178.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 582 kB 00:00 2024-04-25T12:52:18.103 INFO:teuthology.orchestra.run.smithi178.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 517 kB/s | 25 kB 00:00 2024-04-25T12:52:18.128 INFO:teuthology.orchestra.run.smithi178.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T12:52:18.153 INFO:teuthology.orchestra.run.smithi178.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T12:52:18.179 INFO:teuthology.orchestra.run.smithi178.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T12:52:18.204 INFO:teuthology.orchestra.run.smithi178.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 35 kB 00:00 2024-04-25T12:52:18.304 INFO:teuthology.orchestra.run.smithi178.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-25T12:52:18.388 INFO:teuthology.orchestra.run.smithi178.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-04-25T12:52:18.430 INFO:teuthology.orchestra.run.smithi178.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T12:52:18.435 INFO:teuthology.orchestra.run.smithi104.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.9 MB/s | 5.4 MB 00:00 2024-04-25T12:52:18.463 INFO:teuthology.orchestra.run.smithi178.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T12:52:18.488 INFO:teuthology.orchestra.run.smithi178.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T12:52:18.511 INFO:teuthology.orchestra.run.smithi104.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 598 kB/s | 45 kB 00:00 2024-04-25T12:52:18.603 INFO:teuthology.orchestra.run.smithi104.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T12:52:18.605 INFO:teuthology.orchestra.run.smithi178.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 41 MB/s | 5.8 MB 00:00 2024-04-25T12:52:18.639 INFO:teuthology.orchestra.run.smithi178.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-04-25T12:52:18.664 INFO:teuthology.orchestra.run.smithi178.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T12:52:18.686 INFO:teuthology.orchestra.run.smithi104.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.9 MB/s | 161 kB 00:00 2024-04-25T12:52:18.778 INFO:teuthology.orchestra.run.smithi104.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-25T12:52:18.798 INFO:teuthology.orchestra.run.smithi178.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-25T12:52:18.823 INFO:teuthology.orchestra.run.smithi178.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T12:52:18.848 INFO:teuthology.orchestra.run.smithi178.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T12:52:18.853 INFO:teuthology.orchestra.run.smithi104.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.9 MB/s | 298 kB 00:00 2024-04-25T12:52:18.866 INFO:teuthology.orchestra.run.smithi178.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-04-25T12:52:18.882 INFO:teuthology.orchestra.run.smithi178.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 573 kB/s | 19 kB 00:00 2024-04-25T12:52:18.907 INFO:teuthology.orchestra.run.smithi178.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T12:52:18.925 INFO:teuthology.orchestra.run.smithi178.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-04-25T12:52:18.937 INFO:teuthology.orchestra.run.smithi104.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T12:52:19.000 INFO:teuthology.orchestra.run.smithi178.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T12:52:19.012 INFO:teuthology.orchestra.run.smithi104.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T12:52:19.075 INFO:teuthology.orchestra.run.smithi178.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-04-25T12:52:19.109 INFO:teuthology.orchestra.run.smithi178.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-25T12:52:19.134 INFO:teuthology.orchestra.run.smithi178.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T12:52:19.160 INFO:teuthology.orchestra.run.smithi178.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-25T12:52:19.185 INFO:teuthology.orchestra.run.smithi178.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T12:52:19.210 INFO:teuthology.orchestra.run.smithi178.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T12:52:19.235 INFO:teuthology.orchestra.run.smithi178.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T12:52:19.261 INFO:teuthology.orchestra.run.smithi178.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T12:52:19.286 INFO:teuthology.orchestra.run.smithi178.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T12:52:19.429 INFO:teuthology.orchestra.run.smithi104.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.4 MB/s | 3.1 MB 00:00 2024-04-25T12:52:19.436 INFO:teuthology.orchestra.run.smithi178.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 282 kB/s | 42 kB 00:00 2024-04-25T12:52:19.505 INFO:teuthology.orchestra.run.smithi104.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 170 kB 00:00 2024-04-25T12:52:19.571 INFO:teuthology.orchestra.run.smithi104.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 514 kB/s | 34 kB 00:00 2024-04-25T12:52:19.612 INFO:teuthology.orchestra.run.smithi178.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-25T12:52:19.645 INFO:teuthology.orchestra.run.smithi178.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 214 kB/s | 44 kB 00:00 2024-04-25T12:52:19.647 INFO:teuthology.orchestra.run.smithi104.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-25T12:52:19.670 INFO:teuthology.orchestra.run.smithi178.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T12:52:19.696 INFO:teuthology.orchestra.run.smithi178.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-25T12:52:19.721 INFO:teuthology.orchestra.run.smithi178.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-25T12:52:19.746 INFO:teuthology.orchestra.run.smithi178.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T12:52:19.771 INFO:teuthology.orchestra.run.smithi178.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T12:52:19.797 INFO:teuthology.orchestra.run.smithi178.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T12:52:19.822 INFO:teuthology.orchestra.run.smithi178.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T12:52:19.847 INFO:teuthology.orchestra.run.smithi178.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T12:52:19.873 INFO:teuthology.orchestra.run.smithi178.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T12:52:19.906 INFO:teuthology.orchestra.run.smithi178.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T12:52:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.5 MB/s | 3.5 MB 00:00 2024-04-25T12:52:20.306 INFO:teuthology.orchestra.run.smithi178.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 773 kB/s | 309 kB 00:00 2024-04-25T12:52:20.382 INFO:teuthology.orchestra.run.smithi178.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T12:52:20.407 INFO:teuthology.orchestra.run.smithi178.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-25T12:52:20.491 INFO:teuthology.orchestra.run.smithi178.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-04-25T12:52:20.574 INFO:teuthology.orchestra.run.smithi178.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-04-25T12:52:20.709 INFO:teuthology.orchestra.run.smithi038.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 3.7 MB/s | 11 MB 00:02 2024-04-25T12:52:20.766 INFO:teuthology.orchestra.run.smithi178.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 351 kB/s | 67 kB 00:00 2024-04-25T12:52:20.776 INFO:teuthology.orchestra.run.smithi038.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 379 kB/s | 25 kB 00:00 2024-04-25T12:52:20.841 INFO:teuthology.orchestra.run.smithi178.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-25T12:52:20.875 INFO:teuthology.orchestra.run.smithi178.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T12:52:20.958 INFO:teuthology.orchestra.run.smithi178.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-04-25T12:52:21.059 INFO:teuthology.orchestra.run.smithi178.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-25T12:52:21.084 INFO:teuthology.orchestra.run.smithi178.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-04-25T12:52:21.143 INFO:teuthology.orchestra.run.smithi178.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-04-25T12:52:21.164 INFO:teuthology.orchestra.run.smithi104.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-04-25T12:52:21.176 INFO:teuthology.orchestra.run.smithi178.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T12:52:21.243 INFO:teuthology.orchestra.run.smithi178.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T12:52:21.264 INFO:teuthology.orchestra.run.smithi104.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-25T12:52:21.293 INFO:teuthology.orchestra.run.smithi178.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T12:52:21.335 INFO:teuthology.orchestra.run.smithi178.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-25T12:52:21.364 INFO:teuthology.orchestra.run.smithi104.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 492 kB/s | 49 kB 00:00 2024-04-25T12:52:21.377 INFO:teuthology.orchestra.run.smithi178.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 423 kB/s | 35 kB 00:00 2024-04-25T12:52:21.428 INFO:teuthology.orchestra.run.smithi178.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T12:52:21.473 INFO:teuthology.orchestra.run.smithi104.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T12:52:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T12:52:21.581 INFO:teuthology.orchestra.run.smithi104.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 273 kB 00:00 2024-04-25T12:52:21.620 INFO:teuthology.orchestra.run.smithi178.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T12:52:21.670 INFO:teuthology.orchestra.run.smithi178.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-04-25T12:52:21.729 INFO:teuthology.orchestra.run.smithi178.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T12:52:21.762 INFO:teuthology.orchestra.run.smithi178.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-25T12:52:21.765 INFO:teuthology.orchestra.run.smithi104.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 4.1 MB/s | 762 kB 00:00 2024-04-25T12:52:21.821 INFO:teuthology.orchestra.run.smithi178.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-25T12:52:21.854 INFO:teuthology.orchestra.run.smithi178.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T12:52:21.857 INFO:teuthology.orchestra.run.smithi104.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T12:52:22.091 INFO:teuthology.orchestra.run.smithi104.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 80 kB 00:00 2024-04-25T12:52:22.126 INFO:teuthology.orchestra.run.smithi038.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 21 MB 00:09 2024-04-25T12:52:22.257 INFO:teuthology.orchestra.run.smithi104.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-04-25T12:52:22.276 INFO:teuthology.orchestra.run.smithi038.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 225 kB 00:00 2024-04-25T12:52:22.316 INFO:teuthology.orchestra.run.smithi104.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T12:52:22.330 INFO:teuthology.orchestra.run.smithi178.stdout:(129/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 5.1 MB/s | 49 MB 00:09 2024-04-25T12:52:22.355 INFO:teuthology.orchestra.run.smithi178.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 92 kB/s | 46 kB 00:00 2024-04-25T12:52:22.360 INFO:teuthology.orchestra.run.smithi038.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 324 kB/s | 27 kB 00:00 2024-04-25T12:52:22.375 INFO:teuthology.orchestra.run.smithi104.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-25T12:52:22.389 INFO:teuthology.orchestra.run.smithi178.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 1.8 MB/s | 1.0 MB 00:00 2024-04-25T12:52:22.443 INFO:teuthology.orchestra.run.smithi038.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 382 kB/s | 32 kB 00:00 2024-04-25T12:52:22.455 INFO:teuthology.orchestra.run.smithi178.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-25T12:52:22.497 INFO:teuthology.orchestra.run.smithi178.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-25T12:52:22.548 INFO:teuthology.orchestra.run.smithi178.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-25T12:52:22.590 INFO:teuthology.orchestra.run.smithi178.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 990 kB/s | 90 kB 00:00 2024-04-25T12:52:22.640 INFO:teuthology.orchestra.run.smithi178.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-25T12:52:22.682 INFO:teuthology.orchestra.run.smithi178.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-04-25T12:52:22.694 INFO:teuthology.orchestra.run.smithi038.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.9 MB/s | 747 kB 00:00 2024-04-25T12:52:22.732 INFO:teuthology.orchestra.run.smithi178.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-25T12:52:22.766 INFO:teuthology.orchestra.run.smithi178.stdout:(139/152): python3-more-itertools-8.12.0-2.el9. 181 kB/s | 79 kB 00:00 2024-04-25T12:52:22.785 INFO:teuthology.orchestra.run.smithi038.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T12:52:22.791 INFO:teuthology.orchestra.run.smithi178.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-04-25T12:52:22.816 INFO:teuthology.orchestra.run.smithi178.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-25T12:52:22.858 INFO:teuthology.orchestra.run.smithi104.stdout:(42/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 7.6 MB/s | 49 MB 00:06 2024-04-25T12:52:22.869 INFO:teuthology.orchestra.run.smithi038.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T12:52:22.883 INFO:teuthology.orchestra.run.smithi178.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T12:52:22.892 INFO:teuthology.orchestra.run.smithi104.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-04-25T12:52:22.908 INFO:teuthology.orchestra.run.smithi178.stdout:(143/152): python3-typing-extensions-4.4.0-2.el 358 kB/s | 51 kB 00:00 2024-04-25T12:52:22.917 INFO:teuthology.orchestra.run.smithi104.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T12:52:22.942 INFO:teuthology.orchestra.run.smithi178.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 712 kB/s | 90 kB 00:00 2024-04-25T12:52:22.951 INFO:teuthology.orchestra.run.smithi104.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T12:52:22.984 INFO:teuthology.orchestra.run.smithi104.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T12:52:22.993 INFO:teuthology.orchestra.run.smithi178.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T12:52:23.018 INFO:teuthology.orchestra.run.smithi178.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-25T12:52:23.018 INFO:teuthology.orchestra.run.smithi104.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-25T12:52:23.043 INFO:teuthology.orchestra.run.smithi178.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T12:52:23.060 INFO:teuthology.orchestra.run.smithi104.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2024-04-25T12:52:23.094 INFO:teuthology.orchestra.run.smithi038.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.2 MB/s | 500 kB 00:00 2024-04-25T12:52:23.110 INFO:teuthology.orchestra.run.smithi104.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-25T12:52:23.135 INFO:teuthology.orchestra.run.smithi178.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T12:52:23.135 INFO:teuthology.orchestra.run.smithi104.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-25T12:52:23.169 INFO:teuthology.orchestra.run.smithi104.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-04-25T12:52:23.202 INFO:teuthology.orchestra.run.smithi104.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-25T12:52:23.227 INFO:teuthology.orchestra.run.smithi178.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-04-25T12:52:23.236 INFO:teuthology.orchestra.run.smithi104.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-25T12:52:23.403 INFO:teuthology.orchestra.run.smithi104.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T12:52:23.428 INFO:teuthology.orchestra.run.smithi104.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 95 kB/s | 18 kB 00:00 2024-04-25T12:52:23.469 INFO:teuthology.orchestra.run.smithi178.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 7.4 MB/s | 3.1 MB 00:00 2024-04-25T12:52:23.479 INFO:teuthology.orchestra.run.smithi104.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T12:52:23.504 INFO:teuthology.orchestra.run.smithi104.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T12:52:23.519 INFO:teuthology.orchestra.run.smithi178.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-04-25T12:52:23.529 INFO:teuthology.orchestra.run.smithi104.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T12:52:23.562 INFO:teuthology.orchestra.run.smithi104.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T12:52:23.588 INFO:teuthology.orchestra.run.smithi104.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-25T12:52:23.613 INFO:teuthology.orchestra.run.smithi104.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 513 kB/s | 25 kB 00:00 2024-04-25T12:52:23.638 INFO:teuthology.orchestra.run.smithi104.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T12:52:23.672 INFO:teuthology.orchestra.run.smithi104.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T12:52:23.698 INFO:teuthology.orchestra.run.smithi104.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T12:52:23.723 INFO:teuthology.orchestra.run.smithi104.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-25T12:52:23.881 INFO:teuthology.orchestra.run.smithi104.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-25T12:52:23.982 INFO:teuthology.orchestra.run.smithi104.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T12:52:24.023 INFO:teuthology.orchestra.run.smithi104.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T12:52:24.065 INFO:teuthology.orchestra.run.smithi104.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T12:52:24.091 INFO:teuthology.orchestra.run.smithi104.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T12:52:24.316 INFO:teuthology.orchestra.run.smithi104.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-25T12:52:24.349 INFO:teuthology.orchestra.run.smithi104.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-25T12:52:24.375 INFO:teuthology.orchestra.run.smithi104.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T12:52:24.575 INFO:teuthology.orchestra.run.smithi104.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-04-25T12:52:24.600 INFO:teuthology.orchestra.run.smithi104.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T12:52:24.626 INFO:teuthology.orchestra.run.smithi104.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T12:52:24.642 INFO:teuthology.orchestra.run.smithi104.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-04-25T12:52:24.659 INFO:teuthology.orchestra.run.smithi104.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 575 kB/s | 19 kB 00:00 2024-04-25T12:52:24.676 INFO:teuthology.orchestra.run.smithi104.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 5.0 MB/s | 172 kB 00:00 2024-04-25T12:52:24.701 INFO:teuthology.orchestra.run.smithi104.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 828 kB/s | 35 kB 00:00 2024-04-25T12:52:24.894 INFO:teuthology.orchestra.run.smithi104.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-25T12:52:24.927 INFO:teuthology.orchestra.run.smithi104.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-04-25T12:52:24.952 INFO:teuthology.orchestra.run.smithi104.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T12:52:24.978 INFO:teuthology.orchestra.run.smithi104.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T12:52:25.003 INFO:teuthology.orchestra.run.smithi104.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-04-25T12:52:25.036 INFO:teuthology.orchestra.run.smithi104.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T12:52:25.062 INFO:teuthology.orchestra.run.smithi104.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T12:52:25.095 INFO:teuthology.orchestra.run.smithi104.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T12:52:25.120 INFO:teuthology.orchestra.run.smithi038.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.2 MB/s | 17 MB 00:07 2024-04-25T12:52:25.120 INFO:teuthology.orchestra.run.smithi104.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-04-25T12:52:25.146 INFO:teuthology.orchestra.run.smithi104.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T12:52:25.203 INFO:teuthology.orchestra.run.smithi038.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 544 kB/s | 45 kB 00:00 2024-04-25T12:52:25.337 INFO:teuthology.orchestra.run.smithi038.stdout:(20/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 5.4 MB 00:02 2024-04-25T12:52:25.387 INFO:teuthology.orchestra.run.smithi038.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 734 kB/s | 135 kB 00:00 2024-04-25T12:52:25.411 INFO:teuthology.orchestra.run.smithi178.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.4 MB/s | 3.4 MB 00:02 2024-04-25T12:52:25.418 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:52:25.418 INFO:teuthology.orchestra.run.smithi178.stdout:Total 13 MB/s | 222 MB 00:16 2024-04-25T12:52:25.479 INFO:teuthology.orchestra.run.smithi038.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.1 MB/s | 161 kB 00:00 2024-04-25T12:52:25.496 INFO:teuthology.orchestra.run.smithi104.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 119 kB/s | 42 kB 00:00 2024-04-25T12:52:25.588 INFO:teuthology.orchestra.run.smithi038.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 320 kB 00:00 2024-04-25T12:52:25.665 INFO:teuthology.orchestra.run.smithi038.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T12:52:25.691 INFO:teuthology.orchestra.run.smithi038.stdout:(25/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 298 kB 00:00 2024-04-25T12:52:25.738 INFO:teuthology.orchestra.run.smithi104.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-25T12:52:25.741 INFO:teuthology.orchestra.run.smithi038.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T12:52:25.764 INFO:teuthology.orchestra.run.smithi104.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 167 kB/s | 44 kB 00:00 2024-04-25T12:52:25.789 INFO:teuthology.orchestra.run.smithi104.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T12:52:25.816 INFO:teuthology.orchestra.run.smithi104.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 690 kB/s | 36 kB 00:00 2024-04-25T12:52:25.841 INFO:teuthology.orchestra.run.smithi104.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 905 kB/s | 47 kB 00:00 2024-04-25T12:52:25.875 INFO:teuthology.orchestra.run.smithi104.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T12:52:25.900 INFO:teuthology.orchestra.run.smithi104.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-25T12:52:25.934 INFO:teuthology.orchestra.run.smithi104.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T12:52:25.959 INFO:teuthology.orchestra.run.smithi104.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T12:52:25.992 INFO:teuthology.orchestra.run.smithi104.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-25T12:52:26.018 INFO:teuthology.orchestra.run.smithi104.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-04-25T12:52:26.076 INFO:teuthology.orchestra.run.smithi104.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T12:52:26.291 INFO:teuthology.orchestra.run.smithi038.stdout:(27/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.9 MB/s | 49 MB 00:05 2024-04-25T12:52:26.316 INFO:teuthology.orchestra.run.smithi038.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 296 kB/s | 170 kB 00:00 2024-04-25T12:52:26.318 INFO:teuthology.orchestra.run.smithi104.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-25T12:52:26.342 INFO:teuthology.orchestra.run.smithi038.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 678 kB/s | 34 kB 00:00 2024-04-25T12:52:26.343 INFO:teuthology.orchestra.run.smithi104.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-25T12:52:26.377 INFO:teuthology.orchestra.run.smithi104.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-04-25T12:52:26.434 INFO:teuthology.orchestra.run.smithi038.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.2 MB/s | 142 kB 00:00 2024-04-25T12:52:26.525 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-04-25T12:52:26.586 INFO:teuthology.orchestra.run.smithi104.stdout:(107/152): ceph-common-19.0.0-3156.gd69f13d8.el 1.4 MB/s | 21 MB 00:15 2024-04-25T12:52:26.611 INFO:teuthology.orchestra.run.smithi104.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 111 kB/s | 26 kB 00:00 2024-04-25T12:52:26.667 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-04-25T12:52:26.667 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-04-25T12:52:26.668 INFO:teuthology.orchestra.run.smithi038.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 11 MB/s | 3.5 MB 00:00 2024-04-25T12:52:26.669 INFO:teuthology.orchestra.run.smithi104.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T12:52:26.718 INFO:teuthology.orchestra.run.smithi038.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 4.9 MB/s | 249 kB 00:00 2024-04-25T12:52:26.736 INFO:teuthology.orchestra.run.smithi104.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T12:52:26.753 INFO:teuthology.orchestra.run.smithi104.stdout:(111/152): liboath-2.6.7-2.el9.x86_64.rpm 291 kB/s | 49 kB 00:00 2024-04-25T12:52:26.768 INFO:teuthology.orchestra.run.smithi038.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 986 kB/s | 49 kB 00:00 2024-04-25T12:52:26.819 INFO:teuthology.orchestra.run.smithi038.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 330 kB/s | 16 kB 00:00 2024-04-25T12:52:26.837 INFO:teuthology.orchestra.run.smithi104.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-04-25T12:52:26.877 INFO:teuthology.orchestra.run.smithi038.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.6 MB/s | 273 kB 00:00 2024-04-25T12:52:26.904 INFO:teuthology.orchestra.run.smithi104.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-04-25T12:52:26.920 INFO:teuthology.orchestra.run.smithi104.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-25T12:52:26.954 INFO:teuthology.orchestra.run.smithi104.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-25T12:52:26.970 INFO:teuthology.orchestra.run.smithi038.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 8.1 MB/s | 762 kB 00:00 2024-04-25T12:52:26.988 INFO:teuthology.orchestra.run.smithi104.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-25T12:52:27.011 INFO:teuthology.orchestra.run.smithi038.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 440 kB/s | 18 kB 00:00 2024-04-25T12:52:27.013 INFO:teuthology.orchestra.run.smithi104.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 162 kB/s | 15 kB 00:00 2024-04-25T12:52:27.038 INFO:teuthology.orchestra.run.smithi104.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-25T12:52:27.064 INFO:teuthology.orchestra.run.smithi104.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-25T12:52:27.070 INFO:teuthology.orchestra.run.smithi038.stdout:(38/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.2 MB/s | 3.1 MB 00:01 2024-04-25T12:52:27.105 INFO:teuthology.orchestra.run.smithi104.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-04-25T12:52:27.131 INFO:teuthology.orchestra.run.smithi104.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T12:52:27.164 INFO:teuthology.orchestra.run.smithi104.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T12:52:27.226 INFO:teuthology.orchestra.run.smithi104.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T12:52:27.237 INFO:teuthology.orchestra.run.smithi038.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 356 kB/s | 80 kB 00:00 2024-04-25T12:52:27.248 INFO:teuthology.orchestra.run.smithi104.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-25T12:52:27.274 INFO:teuthology.orchestra.run.smithi104.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-25T12:52:27.324 INFO:teuthology.orchestra.run.smithi104.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-25T12:52:27.337 INFO:teuthology.orchestra.run.smithi038.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T12:52:27.357 INFO:teuthology.orchestra.run.smithi104.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2024-04-25T12:52:27.371 INFO:teuthology.orchestra.run.smithi038.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-25T12:52:27.383 INFO:teuthology.orchestra.run.smithi104.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-25T12:52:27.408 INFO:teuthology.orchestra.run.smithi104.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-04-25T12:52:27.421 INFO:teuthology.orchestra.run.smithi038.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-25T12:52:27.521 INFO:teuthology.orchestra.run.smithi038.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-04-25T12:52:27.547 INFO:teuthology.orchestra.run.smithi038.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 853 kB/s | 106 kB 00:00 2024-04-25T12:52:27.552 INFO:teuthology.orchestra.run.smithi104.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 275 kB/s | 46 kB 00:00 2024-04-25T12:52:27.580 INFO:teuthology.orchestra.run.smithi038.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T12:52:27.583 INFO:teuthology.orchestra.run.smithi104.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 450 kB/s | 79 kB 00:00 2024-04-25T12:52:27.606 INFO:teuthology.orchestra.run.smithi038.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T12:52:27.625 INFO:teuthology.orchestra.run.smithi104.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-25T12:52:27.639 INFO:teuthology.orchestra.run.smithi038.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T12:52:27.650 INFO:teuthology.orchestra.run.smithi104.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 584 kB/s | 58 kB 00:00 2024-04-25T12:52:27.676 INFO:teuthology.orchestra.run.smithi104.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-25T12:52:27.698 INFO:teuthology.orchestra.run.smithi038.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 7.4 MB/s | 689 kB 00:00 2024-04-25T12:52:27.701 INFO:teuthology.orchestra.run.smithi104.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-25T12:52:27.735 INFO:teuthology.orchestra.run.smithi104.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-25T12:52:27.760 INFO:teuthology.orchestra.run.smithi104.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-04-25T12:52:27.785 INFO:teuthology.orchestra.run.smithi104.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-04-25T12:52:27.799 INFO:teuthology.orchestra.run.smithi038.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-04-25T12:52:27.810 INFO:teuthology.orchestra.run.smithi104.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T12:52:27.824 INFO:teuthology.orchestra.run.smithi038.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-04-25T12:52:27.836 INFO:teuthology.orchestra.run.smithi104.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-25T12:52:27.858 INFO:teuthology.orchestra.run.smithi038.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 646 kB/s | 21 kB 00:00 2024-04-25T12:52:27.861 INFO:teuthology.orchestra.run.smithi104.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-25T12:52:27.883 INFO:teuthology.orchestra.run.smithi038.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T12:52:27.886 INFO:teuthology.orchestra.run.smithi104.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-25T12:52:27.912 INFO:teuthology.orchestra.run.smithi104.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T12:52:27.916 INFO:teuthology.orchestra.run.smithi038.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 565 kB/s | 33 kB 00:00 2024-04-25T12:52:27.937 INFO:teuthology.orchestra.run.smithi104.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T12:52:27.995 INFO:teuthology.orchestra.run.smithi104.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T12:52:28.021 INFO:teuthology.orchestra.run.smithi104.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-25T12:52:28.046 INFO:teuthology.orchestra.run.smithi104.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T12:52:28.075 INFO:teuthology.orchestra.run.smithi038.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-25T12:52:28.100 INFO:teuthology.orchestra.run.smithi038.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 99 kB/s | 18 kB 00:00 2024-04-25T12:52:28.134 INFO:teuthology.orchestra.run.smithi038.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-25T12:52:28.138 INFO:teuthology.orchestra.run.smithi104.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T12:52:28.161 INFO:teuthology.orchestra.run.smithi038.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T12:52:28.184 INFO:teuthology.orchestra.run.smithi038.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-04-25T12:52:28.243 INFO:teuthology.orchestra.run.smithi038.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T12:52:28.255 INFO:teuthology.orchestra.run.smithi104.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T12:52:28.335 INFO:teuthology.orchestra.run.smithi038.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-25T12:52:28.360 INFO:teuthology.orchestra.run.smithi038.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 218 kB/s | 25 kB 00:00 2024-04-25T12:52:28.388 INFO:teuthology.orchestra.run.smithi104.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 9.2 MB/s | 3.4 MB 00:00 2024-04-25T12:52:28.410 INFO:teuthology.orchestra.run.smithi038.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-04-25T12:52:28.430 INFO:teuthology.orchestra.run.smithi104.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T12:52:28.436 INFO:teuthology.orchestra.run.smithi038.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T12:52:28.442 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-04-25T12:52:28.443 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-04-25T12:52:28.486 INFO:teuthology.orchestra.run.smithi038.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-25T12:52:28.511 INFO:teuthology.orchestra.run.smithi038.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-04-25T12:52:28.737 INFO:teuthology.orchestra.run.smithi038.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-25T12:52:28.914 INFO:teuthology.orchestra.run.smithi104.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.6 MB/s | 3.1 MB 00:00 2024-04-25T12:52:28.918 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:52:28.918 INFO:teuthology.orchestra.run.smithi104.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-25T12:52:28.920 INFO:teuthology.orchestra.run.smithi038.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T12:52:28.962 INFO:teuthology.orchestra.run.smithi038.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-04-25T12:52:28.996 INFO:teuthology.orchestra.run.smithi038.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T12:52:29.021 INFO:teuthology.orchestra.run.smithi038.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T12:52:29.188 INFO:teuthology.orchestra.run.smithi038.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T12:52:29.222 INFO:teuthology.orchestra.run.smithi038.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-04-25T12:52:29.247 INFO:teuthology.orchestra.run.smithi038.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T12:52:29.423 INFO:teuthology.orchestra.run.smithi038.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-25T12:52:29.448 INFO:teuthology.orchestra.run.smithi038.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-04-25T12:52:29.474 INFO:teuthology.orchestra.run.smithi038.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T12:52:29.507 INFO:teuthology.orchestra.run.smithi038.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-04-25T12:52:29.532 INFO:teuthology.orchestra.run.smithi038.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-04-25T12:52:29.558 INFO:teuthology.orchestra.run.smithi038.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T12:52:29.583 INFO:teuthology.orchestra.run.smithi038.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 699 kB/s | 35 kB 00:00 2024-04-25T12:52:29.859 INFO:teuthology.orchestra.run.smithi038.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T12:52:29.909 INFO:teuthology.orchestra.run.smithi038.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 773 kB/s | 250 kB 00:00 2024-04-25T12:52:29.934 INFO:teuthology.orchestra.run.smithi038.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-25T12:52:29.968 INFO:teuthology.orchestra.run.smithi038.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T12:52:29.993 INFO:teuthology.orchestra.run.smithi038.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-04-25T12:52:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-25T12:52:30.027 INFO:teuthology.orchestra.run.smithi038.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T12:52:30.052 INFO:teuthology.orchestra.run.smithi038.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T12:52:30.086 INFO:teuthology.orchestra.run.smithi038.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T12:52:30.111 INFO:teuthology.orchestra.run.smithi038.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 875 kB/s | 51 kB 00:00 2024-04-25T12:52:30.136 INFO:teuthology.orchestra.run.smithi038.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T12:52:30.166 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-25T12:52:30.167 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-25T12:52:30.270 INFO:teuthology.orchestra.run.smithi038.stdout:(91/152): ceph-mgr-diskprediction-local-19.0.0- 1.9 MB/s | 7.4 MB 00:03 2024-04-25T12:52:30.362 INFO:teuthology.orchestra.run.smithi038.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 186 kB/s | 42 kB 00:00 2024-04-25T12:52:30.587 INFO:teuthology.orchestra.run.smithi038.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-25T12:52:30.612 INFO:teuthology.orchestra.run.smithi038.stdout:(94/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 130 kB/s | 44 kB 00:00 2024-04-25T12:52:30.638 INFO:teuthology.orchestra.run.smithi038.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 809 kB/s | 221 kB 00:00 2024-04-25T12:52:30.663 INFO:teuthology.orchestra.run.smithi038.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 481 kB/s | 36 kB 00:00 2024-04-25T12:52:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-04-25T12:52:30.713 INFO:teuthology.orchestra.run.smithi038.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T12:52:30.738 INFO:teuthology.orchestra.run.smithi038.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 774 kB/s | 58 kB 00:00 2024-04-25T12:52:30.764 INFO:teuthology.orchestra.run.smithi038.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-25T12:52:30.789 INFO:teuthology.orchestra.run.smithi038.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 850 kB/s | 64 kB 00:00 2024-04-25T12:52:30.814 INFO:teuthology.orchestra.run.smithi038.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-25T12:52:30.840 INFO:teuthology.orchestra.run.smithi038.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T12:52:30.865 INFO:teuthology.orchestra.run.smithi038.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T12:52:30.982 INFO:teuthology.orchestra.run.smithi038.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-25T12:52:31.099 INFO:teuthology.orchestra.run.smithi038.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T12:52:31.124 INFO:teuthology.orchestra.run.smithi038.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-25T12:52:31.150 INFO:teuthology.orchestra.run.smithi038.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-04-25T12:52:31.175 INFO:teuthology.orchestra.run.smithi038.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-04-25T12:52:31.209 INFO:teuthology.orchestra.run.smithi038.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T12:52:31.250 INFO:teuthology.orchestra.run.smithi038.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T12:52:31.343 INFO:teuthology.orchestra.run.smithi038.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-25T12:52:31.376 INFO:teuthology.orchestra.run.smithi038.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T12:52:31.401 INFO:teuthology.orchestra.run.smithi038.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-04-25T12:52:31.460 INFO:teuthology.orchestra.run.smithi038.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-04-25T12:52:31.486 INFO:teuthology.orchestra.run.smithi038.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T12:52:31.527 INFO:teuthology.orchestra.run.smithi038.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 213 kB/s | 32 kB 00:00 2024-04-25T12:52:31.553 INFO:teuthology.orchestra.run.smithi038.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T12:52:31.578 INFO:teuthology.orchestra.run.smithi038.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T12:52:31.603 INFO:teuthology.orchestra.run.smithi038.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-25T12:52:31.628 INFO:teuthology.orchestra.run.smithi038.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 466 kB/s | 35 kB 00:00 2024-04-25T12:52:31.654 INFO:teuthology.orchestra.run.smithi038.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T12:52:31.687 INFO:teuthology.orchestra.run.smithi038.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T12:52:31.713 INFO:teuthology.orchestra.run.smithi038.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-25T12:52:31.771 INFO:teuthology.orchestra.run.smithi038.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T12:52:31.797 INFO:teuthology.orchestra.run.smithi038.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T12:52:31.822 INFO:teuthology.orchestra.run.smithi038.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2024-04-25T12:52:31.847 INFO:teuthology.orchestra.run.smithi038.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T12:52:31.872 INFO:teuthology.orchestra.run.smithi038.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-25T12:52:31.906 INFO:teuthology.orchestra.run.smithi038.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T12:52:31.925 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-25T12:52:31.926 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-25T12:52:31.931 INFO:teuthology.orchestra.run.smithi038.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 552 kB/s | 46 kB 00:00 2024-04-25T12:52:31.949 INFO:teuthology.orchestra.run.smithi038.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-25T12:52:31.974 INFO:teuthology.orchestra.run.smithi038.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 858 kB/s | 58 kB 00:00 2024-04-25T12:52:31.999 INFO:teuthology.orchestra.run.smithi038.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.9 MB/s | 272 kB 00:00 2024-04-25T12:52:32.025 INFO:teuthology.orchestra.run.smithi038.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-25T12:52:32.050 INFO:teuthology.orchestra.run.smithi038.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T12:52:32.075 INFO:teuthology.orchestra.run.smithi038.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-25T12:52:32.100 INFO:teuthology.orchestra.run.smithi038.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T12:52:32.126 INFO:teuthology.orchestra.run.smithi038.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T12:52:32.151 INFO:teuthology.orchestra.run.smithi038.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-04-25T12:52:32.176 INFO:teuthology.orchestra.run.smithi038.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T12:52:32.201 INFO:teuthology.orchestra.run.smithi038.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-25T12:52:32.227 INFO:teuthology.orchestra.run.smithi038.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T12:52:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T12:52:32.277 INFO:teuthology.orchestra.run.smithi038.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-25T12:52:32.294 INFO:teuthology.orchestra.run.smithi038.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-04-25T12:52:32.320 INFO:teuthology.orchestra.run.smithi038.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T12:52:32.378 INFO:teuthology.orchestra.run.smithi038.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T12:52:32.454 INFO:teuthology.orchestra.run.smithi038.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T12:52:32.562 INFO:teuthology.orchestra.run.smithi038.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T12:52:32.712 INFO:teuthology.orchestra.run.smithi038.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 8.1 MB/s | 3.4 MB 00:00 2024-04-25T12:52:33.496 INFO:teuthology.orchestra.run.smithi038.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.7 MB/s | 3.1 MB 00:01 2024-04-25T12:52:33.501 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:52:33.502 INFO:teuthology.orchestra.run.smithi038.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-25T12:52:34.525 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-04-25T12:52:34.604 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-04-25T12:52:34.605 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:52:34.644 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:52:34.698 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:52:34.731 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:52:34.757 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:52:34.764 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-04-25T12:52:34.764 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-04-25T12:52:34.799 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:52:35.131 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:52:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:35.355 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:35.384 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:52:35.663 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:52:35.710 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:52:35.737 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:35.842 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:35.872 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:52:35.934 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:52:35.956 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:52:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:52:36.182 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:52:36.214 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:52:36.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:52:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:52:36.454 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:52:36.490 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:52:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:52:36.597 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:52:36.641 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:52:36.642 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-04-25T12:52:36.642 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-04-25T12:52:36.673 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:52:36.701 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:52:36.771 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:52:36.824 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:52:36.854 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:52:36.898 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:52:36.975 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:52:37.009 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:52:37.052 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:52:37.081 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:52:37.151 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:52:37.195 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:52:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:52:37.405 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:52:37.456 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:52:37.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:52:38.036 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-25T12:52:38.068 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:52:38.097 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:52:38.125 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:52:38.139 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:52:38.163 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:52:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:52:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:52:38.224 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:52:38.259 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:52:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:52:38.327 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:52:38.379 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:52:38.430 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:52:38.485 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:52:38.558 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:52:38.589 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:52:38.626 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:52:38.647 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:52:38.673 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:52:38.708 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:52:38.751 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:52:38.772 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:38.793 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:52:38.898 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:38.919 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:38.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:52:38.973 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:52:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:52:39.045 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:52:39.140 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:52:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:52:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:52:39.219 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:52:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:52:39.244 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:52:39.265 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:39.269 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:52:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:52:39.368 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:39.391 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:52:39.453 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:52:39.470 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:52:39.475 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:52:39.500 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:52:39.509 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:52:39.537 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:52:39.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:52:39.619 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:52:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:52:39.680 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:52:39.691 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:52:39.715 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:52:39.724 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:52:39.748 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:52:39.790 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:52:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:52:39.832 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:52:39.911 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:52:39.927 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:52:39.963 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:52:39.965 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:52:39.994 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:52:40.000 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:52:40.041 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:52:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:52:40.107 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:52:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:52:40.151 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:52:40.183 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:52:40.211 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:52:40.281 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:52:40.334 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:52:40.372 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:52:40.416 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:52:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:52:40.492 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:52:40.527 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:52:40.570 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:52:40.598 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:52:40.668 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:52:40.738 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:52:40.812 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:52:40.955 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:52:41.007 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:52:41.155 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:52:41.209 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:52:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:52:41.344 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:52:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:52:41.489 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:52:41.528 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:52:41.536 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:52:41.618 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:52:41.648 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:52:41.690 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:52:41.706 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:52:41.728 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:52:41.734 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:52:41.770 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:52:41.811 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:52:41.919 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:52:41.981 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:52:42.036 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:52:42.109 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:52:42.140 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:52:42.177 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:52:42.223 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:52:42.259 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:52:42.302 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:52:42.344 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:52:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:52:42.508 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:52:42.557 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:52:42.582 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:52:42.588 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:52:42.629 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:52:42.674 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-04-25T12:52:42.724 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:52:42.758 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:52:42.763 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:52:42.797 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:52:42.802 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:52:42.828 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:52:42.852 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:52:42.853 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:52:42.928 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:52:42.935 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:52:42.960 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:52:43.003 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:52:43.054 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:52:43.093 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:52:43.121 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:52:43.155 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:52:43.194 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:52:43.226 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:52:43.256 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:52:43.290 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:52:43.323 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:52:43.348 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:52:43.461 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:43.550 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:52:43.583 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:52:43.611 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:52:43.673 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:52:43.687 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:43.730 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:52:43.761 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:52:43.779 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:52:43.808 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:52:43.808 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:52:43.822 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:52:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:52:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:52:43.940 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:52:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:52:43.965 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:52:43.974 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:52:44.023 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:52:44.050 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:44.155 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:52:44.185 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:52:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T12:52:44.269 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:52:44.301 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:52:44.304 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:52:44.471 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:52:44.503 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:52:44.520 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:52:44.610 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:52:44.614 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:52:44.641 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:52:44.677 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:52:44.689 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:52:44.711 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:52:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:52:44.752 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:52:44.767 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:52:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:52:44.863 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:52:44.868 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:52:44.903 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:52:44.917 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:52:44.947 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:52:44.979 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:52:45.005 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:52:45.007 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:52:45.053 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:52:45.078 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:52:45.139 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:52:45.157 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:52:45.168 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:52:45.181 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:52:45.204 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:52:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:52:45.235 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:52:45.257 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:52:45.258 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:52:45.282 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:52:45.315 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T12:52:45.337 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:52:45.358 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T12:52:45.387 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T12:52:45.461 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:52:45.510 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:52:45.589 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:52:45.749 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:52:45.813 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:52:45.984 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:52:46.098 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:52:46.364 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:52:46.452 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:52:46.487 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:52:46.531 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:52:46.569 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T12:52:46.636 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:52:46.800 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:52:46.847 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:52:46.903 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:52:46.977 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:52:47.022 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:52:47.051 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:52:47.099 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:52:47.134 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:52:47.178 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:52:47.220 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:52:47.328 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:52:47.361 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:52:47.408 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:52:47.411 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:52:47.439 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:52:47.440 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:52:47.472 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:52:47.480 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:52:47.577 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:52:47.581 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:47.622 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:52:47.662 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:52:47.687 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:52:47.712 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:52:47.785 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:52:47.790 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:52:47.827 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:52:47.928 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:52:47.943 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:52:47.962 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:52:47.973 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:52:47.990 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:52:48.009 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:52:48.025 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:52:48.064 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:52:48.103 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:52:48.133 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:52:48.168 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:52:48.176 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:48.201 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:52:48.216 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:52:48.243 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:52:48.277 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:52:48.312 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:52:48.313 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:52:48.313 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:52:48.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:52:48.359 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:52:48.368 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:52:48.417 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:52:48.448 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:52:48.490 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:52:48.495 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:52:48.521 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:52:48.592 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:52:48.607 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:52:48.615 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:52:48.667 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:52:49.428 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:52:49.480 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:52:49.514 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:52:49.617 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:52:49.665 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:52:49.762 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:52:49.833 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:52:50.017 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:52:50.047 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:52:50.126 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:52:50.807 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:52:50.893 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:50.924 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:52:50.936 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:51.483 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:51.531 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:52:51.608 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:52:51.608 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:52:51.608 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:52:51.609 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:52:51.640 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:52:52.240 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:52:52.319 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:52:52.348 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:52:52.381 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:52:52.710 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:52:52.744 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:52:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T12:52:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T12:52:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:52:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:52:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:52:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T12:52:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T12:52:55.933 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:55.961 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:56.531 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T12:52:56.565 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:52:56.658 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T12:52:56.658 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:52:56.658 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:52:56.658 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:52:56.692 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:52:59.743 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:52:59.743 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-04-25T12:52:59.744 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:52:59.920 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:52:59.936 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:52:59.936 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:52:59.936 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:52:59.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:01.431 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:53:01.457 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:53:01.457 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:01.457 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:53:01.457 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:53:01.457 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:53:01.457 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:01.698 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:53:01.721 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:53:01.721 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:01.721 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:53:01.721 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:53:01.721 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:53:01.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:02.182 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:53:02.205 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:53:02.205 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:02.205 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:53:02.205 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:53:02.205 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:53:02.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:02.312 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:53:02.361 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:53:02.417 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:53:02.603 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:53:02.650 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:53:02.736 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:53:02.825 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:53:02.893 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:53:02.893 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-04-25T12:53:02.893 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-04-25T12:53:02.894 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-04-25T12:53:02.894 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-04-25T12:53:02.894 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-04-25T12:53:02.894 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-04-25T12:53:02.894 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-04-25T12:53:02.894 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:02.966 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:53:03.008 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:53:03.040 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:53:03.068 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:53:03.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:53:03.090 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:53:03.090 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:03.090 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:53:03.090 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:03.512 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:53:03.541 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:53:03.565 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T12:53:03.618 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T12:53:03.648 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:53:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:53:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:53:04.628 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:04.628 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:53:04.628 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:53:04.628 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:53:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:04.692 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:53:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:53:04.853 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:53:04.875 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:53:04.875 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:04.875 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:53:04.875 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:53:04.876 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:53:04.876 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:05.320 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:53:05.342 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:53:05.342 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:05.343 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:53:05.343 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:53:05.343 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:53:05.343 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:05.451 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:53:05.499 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:53:05.554 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:53:05.719 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:53:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:53:05.758 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:53:05.787 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:53:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:53:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:53:05.953 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:53:06.022 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T12:53:06.052 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:53:06.076 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:53:06.076 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:06.076 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:53:06.076 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:53:06.077 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:53:06.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:06.093 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:53:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:53:06.137 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:53:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:53:06.169 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:53:06.197 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:53:06.236 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T12:53:06.284 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T12:53:06.632 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:53:06.662 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:53:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T12:53:06.738 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T12:53:06.768 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:53:07.215 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:53:07.246 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:53:07.269 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:53:07.269 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:07.270 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:53:07.270 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:53:07.270 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:53:07.270 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:07.314 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:53:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:53:07.336 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:07.336 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:53:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:53:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:53:07.404 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:07.404 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:53:07.404 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:53:07.404 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:53:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:07.707 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:53:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:53:07.730 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:07.730 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:53:07.730 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:53:07.730 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:53:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:07.809 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:53:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-04-25T12:53:08.677 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:08.812 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:53:08.837 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:53:08.858 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:53:08.875 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T12:53:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:53:08.875 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:53:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T12:53:09.139 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:53:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:53:09.161 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:09.162 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:53:09.162 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:53:09.162 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:53:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:53:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:53:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T12:53:09.363 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T12:53:10.288 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:53:10.333 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:53:10.356 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:53:10.356 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:10.356 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:53:10.356 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:53:10.356 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:53:10.357 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:10.401 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:53:10.403 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:53:10.422 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:53:10.422 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:10.423 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:53:10.423 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:10.430 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T12:53:10.430 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:10.430 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:53:10.430 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:53:10.430 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:53:10.430 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:10.468 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:53:10.490 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:53:10.491 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:10.491 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:53:10.491 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:53:10.491 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:53:10.491 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:10.670 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:53:10.693 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T12:53:10.694 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:10.694 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:53:10.694 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:53:10.694 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:53:10.694 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:53:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:53:10.816 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:10.816 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:53:10.816 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:53:10.816 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:53:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:11.154 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:53:11.177 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T12:53:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:53:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:53:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:53:11.178 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:11.285 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:53:11.343 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:53:11.407 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:53:11.611 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:53:11.681 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:53:11.763 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:53:11.862 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:53:12.013 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:53:12.056 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:53:12.088 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:53:12.115 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:53:12.319 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T12:53:12.360 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T12:53:12.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T12:53:12.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:53:12.468 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:53:12.553 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:53:12.582 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T12:53:12.614 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T12:53:12.655 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T12:53:12.678 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T12:53:12.697 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:53:12.713 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:53:12.727 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T12:53:13.360 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:13.400 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:13.420 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:13.807 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T12:53:13.848 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:53:14.258 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T12:53:14.346 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:53:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:53:14.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:53:14.500 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:53:14.500 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:53:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:53:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:53:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:53:14.701 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:53:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:14.862 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T12:53:14.899 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:53:15.008 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T12:53:15.187 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T12:53:15.218 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:53:15.242 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T12:53:15.242 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:15.242 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:53:15.242 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:53:15.242 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:53:15.242 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:15.293 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:53:15.304 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T12:53:15.405 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T12:53:15.406 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T12:53:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T12:53:15.460 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T12:53:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T12:53:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:53:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:53:15.734 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T12:53:15.775 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:53:15.805 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T12:53:16.389 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:16.427 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:16.432 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:53:16.447 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:16.463 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:53:16.486 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T12:53:16.487 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:16.487 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:53:16.487 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:53:16.487 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:53:16.487 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:16.530 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:53:16.553 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T12:53:16.553 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:16.553 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:53:16.553 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:16.597 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:53:16.621 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T12:53:16.621 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:16.621 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:53:16.621 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:53:16.621 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:53:16.621 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:16.940 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:53:16.963 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T12:53:16.963 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:53:16.963 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:53:16.963 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:53:16.964 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:53:16.964 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:17.274 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T12:53:17.365 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:53:17.406 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:53:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:53:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:53:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:53:17.603 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:53:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:53:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:53:17.723 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:53:17.793 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:20.717 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:20.717 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T12:53:20.717 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:53:20.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T12:53:20.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T12:53:20.730 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:53:20.731 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:53:20.732 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:53:20.733 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:53:20.734 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:53:20.735 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:53:20.736 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:53:20.737 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:53:20.738 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:53:20.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:53:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T12:53:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T12:53:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T12:53:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:53:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:53:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T12:53:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:53:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T12:53:22.795 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:53:22.796 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.797 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.798 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.799 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:53:22.800 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:53:22.801 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:53:22.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:53:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:53:22.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:53:22.805 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:53:22.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:53:22.807 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:53:22.808 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:53:22.808 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:53:22.808 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-04-25T12:53:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:53:23.217 DEBUG:teuthology.parallel:result is None 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T12:53:23.377 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T12:53:23.378 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T12:53:23.379 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T12:53:23.390 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T12:53:23.390 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T12:53:23.390 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T12:53:23.390 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:53:23.391 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:53:23.392 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:53:23.393 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:53:23.394 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:53:23.395 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:53:23.396 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:53:23.397 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:53:23.398 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:53:23.399 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:53:23.806 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T12:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:53:23.933 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:53:24.017 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:53:24.046 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:53:24.046 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:53:24.138 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:53:24.171 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:53:24.171 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:53:24.259 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:53:24.331 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:25.502 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.503 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.504 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:53:25.505 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:53:25.506 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:25.507 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.508 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:53:25.509 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:53:25.510 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.511 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:53:25.512 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:53:25.513 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:53:25.514 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:53:25.514 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-25T12:53:25.514 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-25T12:53:25.941 DEBUG:teuthology.parallel:result is None 2024-04-25T12:53:30.059 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T12:53:30.060 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T12:53:30.061 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T12:53:30.062 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T12:53:30.062 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T12:53:30.062 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T12:53:30.062 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T12:53:30.073 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T12:53:30.073 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T12:53:30.073 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T12:53:30.073 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T12:53:30.073 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T12:53:30.073 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:53:30.074 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:53:30.075 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:53:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:53:30.077 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:53:30.078 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:53:30.079 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:53:30.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:53:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T12:53:30.082 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:53:30.082 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T12:53:30.082 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:53:30.082 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:53:30.082 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:53:30.082 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:53:32.113 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.114 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:53:32.115 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.116 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:53:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:53:32.118 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:53:32.119 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:53:32.120 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:53:32.121 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.122 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:53:32.123 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-25T12:53:32.490 DEBUG:teuthology.parallel:result is None 2024-04-25T12:53:32.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:32.721 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:53:32.742 INFO:teuthology.orchestra.run.smithi038.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T12:53:32.743 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T12:53:32.743 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T12:53:32.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:33.006 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:53:33.031 INFO:teuthology.orchestra.run.smithi104.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T12:53:33.031 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T12:53:33.031 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T12:53:33.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:33.256 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:53:33.282 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T12:53:33.282 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T12:53:33.282 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T12:53:33.283 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T12:53:33.284 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:53:33.284 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:53:33.312 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:53:33.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:53:33.343 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:53:33.343 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:53:33.374 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T12:53:33.374 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:53:33.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:53:33.398 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:53:33.464 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:53:33.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:53:33.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:53:33.559 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:53:33.559 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:53:33.587 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:53:33.655 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T12:53:33.655 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:53:33.655 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:53:33.680 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:53:33.745 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:53:33.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:53:33.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:53:33.841 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:53:33.841 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:53:33.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:53:33.937 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T12:53:33.937 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:53:33.937 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:53:33.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:53:34.026 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:53:34.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:53:34.053 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:53:34.119 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:53:34.119 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:53:34.147 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:53:34.215 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T12:53:34.303 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T12:53:34.303 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:34.304 INFO:tasks.cephadm:Cluster fsid is d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:53:34.304 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T12:53:34.304 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.38', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.178'} 2024-04-25T12:53:34.304 INFO:tasks.cephadm:First mon is mon.a on smithi038 2024-04-25T12:53:34.304 INFO:tasks.cephadm:First mgr is x 2024-04-25T12:53:34.304 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T12:53:34.304 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-04-25T12:53:34.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-04-25T12:53:34.356 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-04-25T12:53:34.384 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T12:53:34.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:34.529 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T12:53:34.751 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:53:34.753 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T12:53:34.753 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:35.697 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 12:53 /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:35.697 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.191 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 12:53 /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.191 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.873 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 12:53 /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.873 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.893 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.912 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T12:53:36.950 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T12:53:36.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T12:53:36.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T12:53:36.955 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T12:53:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:53:38.597 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:53:38.673 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:54:07.320 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T12:54:07.321 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T12:54:07.321 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T12:54:07.321 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-04-25T12:54:07.321 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T12:54:07.321 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T12:54:07.321 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-25T12:54:11.958 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-25T12:54:19.455 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-25T12:54:19.455 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T12:54:19.455 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T12:54:19.456 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-04-25T12:54:19.456 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T12:54:19.456 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-04-25T12:54:19.456 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-25T12:54:19.480 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-04-25T12:54:19.513 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-04-25T12:54:19.540 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-04-25T12:54:19.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-04-25T12:54:19.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-04-25T12:54:19.623 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-04-25T12:54:19.650 INFO:tasks.cephadm:Writing seed config... 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T12:54:19.652 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T12:54:19.653 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T12:54:19.654 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T12:54:19.655 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T12:54:19.656 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:54:19.656 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T12:54:19.674 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d3627c2a-0302-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T12:54:19.674 DEBUG:teuthology.orchestra.run.smithi038:mon.a> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T12:54:19.716 DEBUG:teuthology.orchestra.run.smithi038:mgr.x> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T12:54:19.759 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T12:54:19.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.38 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:19.960 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:54:19.960 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'd3627c2a-0302-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.38', '--skip-admin-label'] 2024-04-25T12:54:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T12:54:19.961 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying podman|docker is present... 2024-04-25T12:54:19.992 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T12:54:19.992 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying lvm2 is present... 2024-04-25T12:54:19.992 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying time synchronization is in place... 2024-04-25T12:54:20.000 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T12:54:20.000 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T12:54:20.006 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T12:54:20.006 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-04-25T12:54:20.013 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-04-25T12:54:20.019 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-04-25T12:54:20.019 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-04-25T12:54:20.019 INFO:teuthology.orchestra.run.smithi038.stdout:Repeating the final host check... 2024-04-25T12:54:20.044 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T12:54:20.044 INFO:teuthology.orchestra.run.smithi038.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T12:54:20.045 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl is present 2024-04-25T12:54:20.045 INFO:teuthology.orchestra.run.smithi038.stdout:lvcreate is present 2024-04-25T12:54:20.051 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T12:54:20.051 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T12:54:20.057 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T12:54:20.057 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-04-25T12:54:20.064 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-04-25T12:54:20.070 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-04-25T12:54:20.070 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-04-25T12:54:20.070 INFO:teuthology.orchestra.run.smithi038.stdout:Host looks OK 2024-04-25T12:54:20.070 INFO:teuthology.orchestra.run.smithi038.stdout:Cluster fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:20.071 INFO:teuthology.orchestra.run.smithi038.stdout:Acquiring lock 139708942753744 on /run/cephadm/d3627c2a-0302-11ef-bc93-c7b262605968.lock 2024-04-25T12:54:20.071 INFO:teuthology.orchestra.run.smithi038.stdout:Lock 139708942753744 acquired on /run/cephadm/d3627c2a-0302-11ef-bc93-c7b262605968.lock 2024-04-25T12:54:20.071 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 3300 ... 2024-04-25T12:54:20.071 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 6789 ... 2024-04-25T12:54:20.071 INFO:teuthology.orchestra.run.smithi038.stdout:Base mon IP(s) is [172.21.15.38:3300, 172.21.15.38:6789], mon addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-04-25T12:54:20.075 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-04-25T12:54:20.075 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-04-25T12:54:20.078 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T12:54:20.078 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T12:54:20.078 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T12:54:20.080 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T12:54:20.080 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T12:54:20.080 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T12:54:20.081 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T12:54:20.081 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T12:54:20.081 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T12:54:20.082 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-04-25T12:54:20.082 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-04-25T12:54:20.082 INFO:teuthology.orchestra.run.smithi038.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T12:54:20.082 INFO:teuthology.orchestra.run.smithi038.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T12:54:20.082 INFO:teuthology.orchestra.run.smithi038.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T12:54:20.514 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T12:54:23.303 INFO:teuthology.orchestra.run.smithi038.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T12:54:23.304 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T12:54:23.304 INFO:teuthology.orchestra.run.smithi038.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T12:54:32.486 INFO:teuthology.orchestra.run.smithi038.stdout:stat: stdout 167 167 2024-04-25T12:54:32.487 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial keys... 2024-04-25T12:54:34.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQCJUipmLbWxKhAA2FhwkFDre3YooGKgvldTbw== 2024-04-25T12:54:35.217 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQCKUipm766gJhAAjbxFAtQ2tFVMpmp6cRkXpQ== 2024-04-25T12:54:36.234 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQCLUipmylKkIRAAGaPFGHB4z9Kp1mfWdGOz1Q== 2024-04-25T12:54:36.235 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial monmap... 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool for a [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: set fsid to d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:54:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mon... 2024-04-25T12:54:38.563 INFO:teuthology.orchestra.run.smithi038.stdout:create mon.a on 2024-04-25T12:54:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T12:54:38.997 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T12:54:39.483 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d3627c2a-0302-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d3627c2a-0302-11ef-bc93-c7b262605968.target. 2024-04-25T12:54:39.483 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d3627c2a-0302-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d3627c2a-0302-11ef-bc93-c7b262605968.target. 2024-04-25T12:54:39.780 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a 2024-04-25T12:54:39.780 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T12:54:39.952 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d3627c2a-0302-11ef-bc93-c7b262605968.target.wants/ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d3627c2a-0302-11ef-bc93-c7b262605968@.service. 2024-04-25T12:54:40.478 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:40 smithi038 ceph-mon[29029]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T12:54:40.492 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:54:40.493 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-04-25T12:54:40.500 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:54:40.500 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-04-25T12:54:40.500 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-04-25T12:54:40.500 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T12:54:40.500 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon to start... 2024-04-25T12:54:40.500 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon... 2024-04-25T12:54:40.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:40 smithi038 bash[29010]: 2b1bbb5c5a0b1754f09206230b2ef79f15880cccc82d278c005db47a15954366 2024-04-25T12:54:40.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:40 smithi038 systemd[1]: Started Ceph mon.a for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: monmap epoch 1 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: last_changed 2024-04-25T12:54:36.611286+0000 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: min_mon_release 19 (squid) 2024-04-25T12:54:41.629 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: election_strategy: 1 2024-04-25T12:54:41.630 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:54:41.630 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: fsmap 2024-04-25T12:54:41.630 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T12:54:41.630 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: mgrmap e1: no daemons active 2024-04-25T12:54:41.630 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:41 smithi038 ceph-mon[29029]: from='client.? 172.21.15.38:0/1580653026' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T12:54:41.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout id: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout services: 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.602263s) 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout data: 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T12:54:41.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.967+0000 7fc5198a5700 1 Processor -- start 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.967+0000 7fc5198a5700 1 -- start start 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.967+0000 7fc5198a5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc51410b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.967+0000 7fc5198a5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc51407be40 con 0x7fc514109070 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.968+0000 7fc512ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc51410b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.968+0000 7fc512ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc51410b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59528/0 (socket says 172.21.15.38:59528) 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.968+0000 7fc512ffd700 1 -- 172.21.15.38:0/2934895176 learned_addr learned my addr 172.21.15.38:0/2934895176 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.968+0000 7fc512ffd700 1 -- 172.21.15.38:0/2934895176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51410b9a0 con 0x7fc514109070 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.968+0000 7fc512ffd700 1 --2- 172.21.15.38:0/2934895176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc51410b460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc5040098a0 tx=0x7fc504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c20e40eb641e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:41.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.968+0000 7fc511ffb700 1 -- 172.21.15.38:0/2934895176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc504004160 con 0x7fc514109070 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc511ffb700 1 -- 172.21.15.38:0/2934895176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc504004300 con 0x7fc514109070 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc511ffb700 1 -- 172.21.15.38:0/2934895176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5040036a0 con 0x7fc514109070 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 -- 172.21.15.38:0/2934895176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 msgr2=0x7fc51410b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 --2- 172.21.15.38:0/2934895176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc51410b460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc5040098a0 tx=0x7fc504009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 -- 172.21.15.38:0/2934895176 shutdown_connections 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 --2- 172.21.15.38:0/2934895176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc51410b460 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 -- 172.21.15.38:0/2934895176 >> 172.21.15.38:0/2934895176 conn(0x7fc514102a40 msgr2=0x7fc514104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 -- 172.21.15.38:0/2934895176 shutdown_connections 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.969+0000 7fc5198a5700 1 -- 172.21.15.38:0/2934895176 wait complete. 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc5198a5700 1 Processor -- start 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc5198a5700 1 -- start start 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc5198a5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc5141088a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:41.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc5198a5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410be10 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc512ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc5141088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc512ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc5141088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59538/0 (socket says 172.21.15.38:59538) 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc512ffd700 1 -- 172.21.15.38:0/1580653026 learned_addr learned my addr 172.21.15.38:0/1580653026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.970+0000 7fc512ffd700 1 -- 172.21.15.38:0/1580653026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc514108de0 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc512ffd700 1 --2- 172.21.15.38:0/1580653026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc5141088a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc504009fb0 tx=0x7fc504004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc504003de0 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc504011630 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc514107020 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc504011930 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc514107500 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc504011ad0 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.971+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc50401b450 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.972+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5140640a0 con 0x7fc514109070 2024-04-25T12:54:41.641 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:40.974+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc504004260 con 0x7fc514109070 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.048+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc514107780 con 0x7fc514109070 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.048+0000 7fc5188a3700 1 -- 172.21.15.38:0/1580653026 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc504025480 con 0x7fc514109070 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.049+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 msgr2=0x7fc5141088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.049+0000 7fc5198a5700 1 --2- 172.21.15.38:0/1580653026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc5141088a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc504009fb0 tx=0x7fc504004750 comp rx=0 tx=0).stop 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.049+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 shutdown_connections 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.049+0000 7fc5198a5700 1 --2- 172.21.15.38:0/1580653026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc514109070 0x7fc5141088a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.049+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 >> 172.21.15.38:0/1580653026 conn(0x7fc514102a40 msgr2=0x7fc5141038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.050+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 shutdown_connections 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:41.050+0000 7fc5198a5700 1 -- 172.21.15.38:0/1580653026 wait complete. 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:mon is available 2024-04-25T12:54:41.642 INFO:teuthology.orchestra.run.smithi038.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T12:54:42.786 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:42 smithi038 ceph-mon[29029]: from='client.? 172.21.15.38:0/1258578332' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T12:54:42.786 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:42 smithi038 ceph-mon[29029]: from='client.? 172.21.15.38:0/1258578332' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T12:54:42.795 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.086+0000 7fbbe48e4700 1 Processor -- start 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.086+0000 7fbbe48e4700 1 -- start start 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.086+0000 7fbbe48e4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe01089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:42.796 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.086+0000 7fbbe48e4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0108fb0 con 0x7fbbe01085c0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbde59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe01089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbde59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe01089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59548/0 (socket says 172.21.15.38:59548) 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbde59c700 1 -- 172.21.15.38:0/3326358763 learned_addr learned my addr 172.21.15.38:0/3326358763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbde59c700 1 -- 172.21.15.38:0/3326358763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0109770 con 0x7fbbe01085c0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbde59c700 1 --2- 172.21.15.38:0/3326358763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe01089e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00098a0 tx=0x7fbbd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23cae89d2ffd06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbdd59a700 1 -- 172.21.15.38:0/3326358763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd0004160 con 0x7fbbe01085c0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbdd59a700 1 -- 172.21.15.38:0/3326358763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0004300 con 0x7fbbe01085c0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.087+0000 7fbbdd59a700 1 -- 172.21.15.38:0/3326358763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd00036a0 con 0x7fbbe01085c0 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 -- 172.21.15.38:0/3326358763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 msgr2=0x7fbbe01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 --2- 172.21.15.38:0/3326358763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe01089e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00098a0 tx=0x7fbbd0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 -- 172.21.15.38:0/3326358763 shutdown_connections 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 --2- 172.21.15.38:0/3326358763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe01089e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 -- 172.21.15.38:0/3326358763 >> 172.21.15.38:0/3326358763 conn(0x7fbbe0103be0 msgr2=0x7fbbe0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 -- 172.21.15.38:0/3326358763 shutdown_connections 2024-04-25T12:54:42.797 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.088+0000 7fbbe48e4700 1 -- 172.21.15.38:0/3326358763 wait complete. 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbe48e4700 1 Processor -- start 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbe48e4700 1 -- start start 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbe48e4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbe48e4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0109ca0 con 0x7fbbe01085c0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbde59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbde59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe019bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59552/0 (socket says 172.21.15.38:59552) 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbde59c700 1 -- 172.21.15.38:0/1258578332 learned_addr learned my addr 172.21.15.38:0/1258578332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.089+0000 7fbbde59c700 1 -- 172.21.15.38:0/1258578332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe019c0d0 con 0x7fbbe01085c0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbde59c700 1 --2- 172.21.15.38:0/1258578332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe019bb90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0000c00 tx=0x7fbbd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd0003db0 con 0x7fbbe01085c0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0011630 con 0x7fbbe01085c0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe019c360 con 0x7fbbe01085c0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd0011950 con 0x7fbbe01085c0 2024-04-25T12:54:42.798 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe019f050 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbbd0011af0 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.090+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbbd001c790 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.091+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe00640a0 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.093+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbbd0004260 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.164+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbbe002e190 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.168+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fbbd001ca30 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbcf7fe700 1 -- 172.21.15.38:0/1258578332 <== mon.0 v2:172.21.15.38:3300/0 8 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fbbd001b900 con 0x7fbbe01085c0 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 msgr2=0x7fbbe019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 --2- 172.21.15.38:0/1258578332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe019bb90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0000c00 tx=0x7fbbd0004750 comp rx=0 tx=0).stop 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 shutdown_connections 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 --2- 172.21.15.38:0/1258578332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe01085c0 0x7fbbe019bb90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:42.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 >> 172.21.15.38:0/1258578332 conn(0x7fbbe0103be0 msgr2=0x7fbbe0192820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:42.800 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 shutdown_connections 2024-04-25T12:54:42.800 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:42.169+0000 7fbbe48e4700 1 -- 172.21.15.38:0/1258578332 wait complete. 2024-04-25T12:54:42.800 INFO:teuthology.orchestra.run.smithi038.stdout:Generating new minimal ceph.conf... 2024-04-25T12:54:43.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:43 smithi038 ceph-mon[29029]: from='client.? 172.21.15.38:0/3925028576' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:54:43.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.220+0000 7fc97d869700 1 Processor -- start 2024-04-25T12:54:43.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.220+0000 7fc97d869700 1 -- start start 2024-04-25T12:54:43.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc97d869700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc978107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:43.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc97d869700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc97807be40 con 0x7fc978106e60 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc976ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc978107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc976ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc978107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59562/0 (socket says 172.21.15.38:59562) 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc976ffd700 1 -- 172.21.15.38:0/3313401801 learned_addr learned my addr 172.21.15.38:0/3313401801 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc976ffd700 1 -- 172.21.15.38:0/3313401801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9781077c0 con 0x7fc978106e60 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc976ffd700 1 --2- 172.21.15.38:0/3313401801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc978107280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098a0 tx=0x7fc96c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb8e19488e0e399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc975ffb700 1 -- 172.21.15.38:0/3313401801 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c004160 con 0x7fc978106e60 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc975ffb700 1 -- 172.21.15.38:0/3313401801 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc96c004300 con 0x7fc978106e60 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.221+0000 7fc975ffb700 1 -- 172.21.15.38:0/3313401801 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c011b20 con 0x7fc978106e60 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 -- 172.21.15.38:0/3313401801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 msgr2=0x7fc978107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:43.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 --2- 172.21.15.38:0/3313401801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc978107280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098a0 tx=0x7fc96c009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 -- 172.21.15.38:0/3313401801 shutdown_connections 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 --2- 172.21.15.38:0/3313401801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc978107280 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 -- 172.21.15.38:0/3313401801 >> 172.21.15.38:0/3313401801 conn(0x7fc978102a40 msgr2=0x7fc978104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 -- 172.21.15.38:0/3313401801 shutdown_connections 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.222+0000 7fc97d869700 1 -- 172.21.15.38:0/3313401801 wait complete. 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc97d869700 1 Processor -- start 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc97d869700 1 -- start start 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc97d869700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc97819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc97d869700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc978108850 con 0x7fc978106e60 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc976ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc97819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc976ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc97819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59572/0 (socket says 172.21.15.38:59572) 2024-04-25T12:54:43.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc976ffd700 1 -- 172.21.15.38:0/3925028576 learned_addr learned my addr 172.21.15.38:0/3925028576 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.223+0000 7fc976ffd700 1 -- 172.21.15.38:0/3925028576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97819eb50 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc976ffd700 1 --2- 172.21.15.38:0/3925028576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc97819e610 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006af0 tx=0x7fc96c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c0039a0 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc96c003b40 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c019670 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97819ede0 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97819f240 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc96c019c70 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.224+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc96c024730 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.225+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9780640a0 con 0x7fc978106e60 2024-04-25T12:54:43.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.227+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc96c043e50 con 0x7fc978106e60 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.298+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc97802e280 con 0x7fc978106e60 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.298+0000 7fc97c867700 1 -- 172.21.15.38:0/3925028576 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fc96c032020 con 0x7fc978106e60 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 msgr2=0x7fc97819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 --2- 172.21.15.38:0/3925028576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc97819e610 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006af0 tx=0x7fc96c011ce0 comp rx=0 tx=0).stop 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 shutdown_connections 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 --2- 172.21.15.38:0/3925028576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc978106e60 0x7fc97819e610 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 >> 172.21.15.38:0/3925028576 conn(0x7fc978102a40 msgr2=0x7fc9781952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 shutdown_connections 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:43.299+0000 7fc97d869700 1 -- 172.21.15.38:0/3925028576 wait complete. 2024-04-25T12:54:43.793 INFO:teuthology.orchestra.run.smithi038.stdout:Restarting the monitor... 2024-04-25T12:54:44.065 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:43 smithi038 systemd[1]: Stopping Ceph mon.a for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T12:54:44.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a[29025]: 2024-04-25T12:54:44.066+0000 7fe9d344f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:54:44.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a[29025]: 2024-04-25T12:54:44.066+0000 7fe9d344f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T12:54:44.631 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 podman[29273]: 2024-04-25 12:54:44.333565501 +0000 UTC m=+0.371859309 container died 2b1bbb5c5a0b1754f09206230b2ef79f15880cccc82d278c005db47a15954366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:54:44.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 podman[29273]: 2024-04-25 12:54:44.633830341 +0000 UTC m=+0.672124130 container cleanup 2b1bbb5c5a0b1754f09206230b2ef79f15880cccc82d278c005db47a15954366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T12:54:44.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 bash[29273]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a 2024-04-25T12:54:44.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 podman[29287]: 2024-04-25 12:54:44.851410754 +0000 UTC m=+0.518962145 container remove 2b1bbb5c5a0b1754f09206230b2ef79f15880cccc82d278c005db47a15954366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T12:54:45.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T12:54:45.414 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 systemd[1]: Stopped Ceph mon.a for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:54:45.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:44 smithi038 systemd[1]: Starting Ceph mon.a for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T12:54:45.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 podman[29381]: 2024-04-25 12:54:45.093802892 +0000 UTC m=+0.019564592 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 podman[29381]: 2024-04-25 12:54:45.327210169 +0000 UTC m=+0.252971846 container create 9c0a6e656491401ab3e9a055a8fbebadba01b84c116862882d1014210bd724bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 podman[29381]: 2024-04-25 12:54:45.546721293 +0000 UTC m=+0.472482973 container init 9c0a6e656491401ab3e9a055a8fbebadba01b84c116862882d1014210bd724bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 podman[29381]: 2024-04-25 12:54:45.549814056 +0000 UTC m=+0.475575734 container start 9c0a6e656491401ab3e9a055a8fbebadba01b84c116862882d1014210bd724bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: pidfile_write: ignore empty --pid-file 2024-04-25T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: load: jerasure load: lrc 2024-04-25T12:54:45.666 INFO:teuthology.orchestra.run.smithi038.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T12:54:45.890 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: RocksDB version: 7.9.2 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Git sha 0 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: DB SUMMARY 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: DB Session ID: C898RS1ER0AFFTM6IW3L 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: CURRENT file: CURRENT 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105602 ; 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.error_if_exists: 0 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.create_if_missing: 0 2024-04-25T12:54:45.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.paranoid_checks: 1 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.env: 0x55d89e3a7cc0 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.info_log: 0x55d89f69bbe0 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.statistics: (nil) 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.use_fsync: 0 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_log_file_size: 0 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T12:54:45.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.allow_fallocate: 1 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.use_direct_reads: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.db_log_dir: 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.wal_dir: 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T12:54:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.write_buffer_manager: 0x55d89f62f0e0 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T12:54:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.unordered_write: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.row_cache: None 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.wal_filter: None 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.two_write_queues: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.wal_compression: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.atomic_flush: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T12:54:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.log_readahead_size: 0 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_background_jobs: 2 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_background_compactions: -1 2024-04-25T12:54:45.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_subcompactions: 1 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_open_files: -1 2024-04-25T12:54:45.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T12:54:45.899 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T12:54:45.899 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T12:54:45.899 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_background_flushes: -1 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Compression algorithms supported: 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kZSTD supported: 0 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kXpressCompression supported: 0 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kZlibCompression supported: 1 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kSnappyCompression supported: 1 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kLZ4Compression supported: 1 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: kBZip2Compression supported: 0 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T12:54:45.900 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.merge_operator: 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_filter: None 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d89f69bdc0) 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T12:54:45.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_type: 0 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_index_type: 0 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_shortening: 1 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: checksum: 4 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: no_block_cache: 0 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache: 0x55d89f6611f0 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_options: 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: capacity : 536870912 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_shard_bits : 4 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: strict_capacity_limit : 0 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-04-25T12:54:45.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_compressed: (nil) 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: persistent_cache: (nil) 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size: 4096 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size_deviation: 10 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_restart_interval: 16 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_block_restart_interval: 1 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: metadata_block_size: 4096 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: partition_filters: 0 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: use_delta_encoding: 1 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: filter_policy: bloomfilter 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: whole_key_filtering: 1 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: verify_compression: 0 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-04-25T12:54:45.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: format_version: 5 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_index_compression: 1 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_align: 0 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout: max_auto_readahead_size: 262144 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout: prepopulate_block_cache: 0 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout: initial_auto_readahead_size: 8192 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression: NoCompression 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.num_levels: 7 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T12:54:45.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T12:54:45.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T12:54:45.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T12:54:45.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.inplace_update_support: 0 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T12:54:45.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.bloom_locality: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.max_successive_merges: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.ttl: 2592000 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.enable_blob_files: false 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.min_blob_size: 0 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T12:54:45.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 486b32e8-da7d-49e8-9938-899c33e9b613 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049685580625, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049685581762, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100689, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98573, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14288, "raw_average_key_size": 49, "raw_value_size": 90829, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714049685, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "486b32e8-da7d-49e8-9938-899c33e9b613", "db_session_id": "C898RS1ER0AFFTM6IW3L", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T12:54:45.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049685581848, "job": 1, "event": "recovery_finished"} 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d89f776000 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: DB pointer 0x55d89f760000 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** DB Stats ** 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:54:45.911 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: L0 2/0 100.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Sum 2/0 100.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:54:45.912 INFO:journalctl@ceph.mon.a.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative compaction: 0.00 GB write, 17.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval compaction: 0.00 GB write, 17.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T12:54:45.913 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache BinnedLRUCache@0x55d89f6611f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: starting mon.a rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???) e1 preinit fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).mds e1 new map 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).mds e1 print_map 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: e1 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: legacy client fscid: -1 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-04-25T12:54:45.914 INFO:journalctl@ceph.mon.a.smithi038.stdout: No filesystems configured 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: monmap epoch 1 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: last_changed 2024-04-25T12:54:36.611286+0000 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: min_mon_release 19 (squid) 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: election_strategy: 1 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: fsmap 2024-04-25T12:54:45.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T12:54:45.916 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 ceph-mon[29400]: mgrmap e1: no daemons active 2024-04-25T12:54:45.916 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 bash[29381]: 9c0a6e656491401ab3e9a055a8fbebadba01b84c116862882d1014210bd724bb 2024-04-25T12:54:45.916 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:45 smithi038 systemd[1]: Started Ceph mon.a for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:54:46.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.174+0000 7f9b329ca700 1 Processor -- start 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.174+0000 7f9b329ca700 1 -- start start 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.174+0000 7f9b329ca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.175+0000 7f9b329ca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c109040 con 0x7f9b2c108650 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.175+0000 7f9b2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.175+0000 7f9b2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59582/0 (socket says 172.21.15.38:59582) 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.175+0000 7f9b2bfff700 1 -- 172.21.15.38:0/3379004952 learned_addr learned my addr 172.21.15.38:0/3379004952 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.175+0000 7f9b2bfff700 1 -- 172.21.15.38:0/3379004952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c109800 con 0x7f9b2c108650 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.175+0000 7f9b2bfff700 1 --2- 172.21.15.38:0/3379004952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c108a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0098a0 tx=0x7f9b1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e62a96925619c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b2affd700 1 -- 172.21.15.38:0/3379004952 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c004160 con 0x7f9b2c108650 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b2affd700 1 -- 172.21.15.38:0/3379004952 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f9b1c004300 con 0x7f9b2c108650 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b2affd700 1 -- 172.21.15.38:0/3379004952 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c011b20 con 0x7f9b2c108650 2024-04-25T12:54:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 -- 172.21.15.38:0/3379004952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 msgr2=0x7f9b2c108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 --2- 172.21.15.38:0/3379004952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c108a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0098a0 tx=0x7f9b1c009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 -- 172.21.15.38:0/3379004952 shutdown_connections 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 --2- 172.21.15.38:0/3379004952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c108a70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 -- 172.21.15.38:0/3379004952 >> 172.21.15.38:0/3379004952 conn(0x7f9b2c103cb0 msgr2=0x7f9b2c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 -- 172.21.15.38:0/3379004952 shutdown_connections 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.176+0000 7f9b329ca700 1 -- 172.21.15.38:0/3379004952 wait complete. 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b329ca700 1 Processor -- start 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b329ca700 1 -- start start 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b329ca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b329ca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c10cbb0 con 0x7f9b2c108650 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59594/0 (socket says 172.21.15.38:59594) 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.177+0000 7f9b2bfff700 1 -- 172.21.15.38:0/2931073867 learned_addr learned my addr 172.21.15.38:0/2931073867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:46.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b2bfff700 1 -- 172.21.15.38:0/2931073867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c19eb90 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b2bfff700 1 --2- 172.21.15.38:0/2931073867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c19e650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006af0 tx=0x7f9b1c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c003b90 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f9b1c003d30 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c19ee20 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c0197c0 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c19f280 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b1c022020 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.178+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b1c02c600 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.179+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2c0640a0 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.182+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9b1c0042c0 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.253+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9b2c1a1b90 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.255+0000 7f9b297fa700 1 -- 172.21.15.38:0/2931073867 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9b1c01ac50 con 0x7f9b2c108650 2024-04-25T12:54:46.836 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.255+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 msgr2=0x7f9b2c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.255+0000 7f9b329ca700 1 --2- 172.21.15.38:0/2931073867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c19e650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006af0 tx=0x7f9b1c011ce0 comp rx=0 tx=0).stop 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.256+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 shutdown_connections 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.256+0000 7f9b329ca700 1 --2- 172.21.15.38:0/2931073867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b2c108650 0x7f9b2c19e650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.256+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 >> 172.21.15.38:0/2931073867 conn(0x7f9b2c103cb0 msgr2=0x7f9b2c104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.256+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 shutdown_connections 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:46.256+0000 7f9b329ca700 1 -- 172.21.15.38:0/2931073867 wait complete. 2024-04-25T12:54:46.837 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T12:54:46.859 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:54:46.859 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mgr... 2024-04-25T12:54:46.859 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T12:54:46.860 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T12:54:47.117 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x 2024-04-25T12:54:47.118 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T12:54:47.271 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d3627c2a-0302-11ef-bc93-c7b262605968.target.wants/ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d3627c2a-0302-11ef-bc93-c7b262605968@.service. 2024-04-25T12:54:47.503 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:47 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2931073867' entity='client.admin' 2024-04-25T12:54:47.504 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 systemd[1]: Starting Ceph mgr.x for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T12:54:47.788 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 podman[29616]: 2024-04-25 12:54:47.505581285 +0000 UTC m=+0.111263646 container create e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T12:54:47.788 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 podman[29616]: 2024-04-25 12:54:47.412683191 +0000 UTC m=+0.018365525 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T12:54:47.788 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 podman[29616]: 2024-04-25 12:54:47.707698441 +0000 UTC m=+0.313380751 container init e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:54:47.788 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 podman[29616]: 2024-04-25 12:54:47.710260343 +0000 UTC m=+0.315942651 container start e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T12:54:47.788 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 bash[29616]: e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 2024-04-25T12:54:47.802 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:54:47.802 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-04-25T12:54:47.809 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:54:47.809 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-04-25T12:54:47.809 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-04-25T12:54:47.809 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T12:54:47.818 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:54:47.818 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-04-25T12:54:47.825 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:54:47.826 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-04-25T12:54:47.826 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-04-25T12:54:47.826 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T12:54:47.826 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr to start... 2024-04-25T12:54:47.826 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr... 2024-04-25T12:54:48.057 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 systemd[1]: Started Ceph mgr.x for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:54:48.057 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:47.866+0000 7fe730042200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:54:48.057 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:48.059+0000 7fe730042200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:54:48.317 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:48.166+0000 7fe730042200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:54:48.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/375095380' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:54:48.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:48.498+0000 7fe730042200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:54:49.003 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:48.586+0000 7fe730042200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:54:49.015 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:49.015 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:54:49.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:54:49.017 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:54:49.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:54:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:54:40.452239+0000", 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.355+0000 7fc5ac421700 1 Processor -- start 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.355+0000 7fc5ac421700 1 -- start start 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.356+0000 7fc5ac421700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a4108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.356+0000 7fc5ac421700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4109000 con 0x7fc5a4108610 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.356+0000 7fc5aa1bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a4108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:49.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.356+0000 7fc5aa1bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a4108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59618/0 (socket says 172.21.15.38:59618) 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.356+0000 7fc5aa1bd700 1 -- 172.21.15.38:0/826968025 learned_addr learned my addr 172.21.15.38:0/826968025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.356+0000 7fc5aa1bd700 1 -- 172.21.15.38:0/826968025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a41097c0 con 0x7fc5a4108610 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.357+0000 7fc5aa1bd700 1 --2- 172.21.15.38:0/826968025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a4108a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00098a0 tx=0x7fc5a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b045520c6d59a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.357+0000 7fc5a91bb700 1 -- 172.21.15.38:0/826968025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a0004160 con 0x7fc5a4108610 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.357+0000 7fc5a91bb700 1 -- 172.21.15.38:0/826968025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc5a0004300 con 0x7fc5a4108610 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.357+0000 7fc5a91bb700 1 -- 172.21.15.38:0/826968025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a0011b20 con 0x7fc5a4108610 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 -- 172.21.15.38:0/826968025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 msgr2=0x7fc5a4108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 --2- 172.21.15.38:0/826968025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a4108a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00098a0 tx=0x7fc5a0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 -- 172.21.15.38:0/826968025 shutdown_connections 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 --2- 172.21.15.38:0/826968025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a4108a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 -- 172.21.15.38:0/826968025 >> 172.21.15.38:0/826968025 conn(0x7fc5a4103c10 msgr2=0x7fc5a4106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 -- 172.21.15.38:0/826968025 shutdown_connections 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 -- 172.21.15.38:0/826968025 wait complete. 2024-04-25T12:54:49.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 Processor -- start 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.358+0000 7fc5ac421700 1 -- start start 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5ac421700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5ac421700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410cb70 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5aa1bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5aa1bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59626/0 (socket says 172.21.15.38:59626) 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5aa1bd700 1 -- 172.21.15.38:0/375095380 learned_addr learned my addr 172.21.15.38:0/375095380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5aa1bd700 1 -- 172.21.15.38:0/375095380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a407e0a0 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5aa1bd700 1 --2- 172.21.15.38:0/375095380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a407db60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00117e0 tx=0x7fc5a0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a0003b20 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc5a0003cc0 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.359+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a407c290 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.360+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a407c750 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.360+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a0019870 con 0x7fc5a4108610 2024-04-25T12:54:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.360+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5a002d430 con 0x7fc5a4108610 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.360+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc5a0024860 con 0x7fc5a4108610 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.361+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc588005360 con 0x7fc5a4108610 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.363+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc5a0011e20 con 0x7fc5a4108610 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.437+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc588005920 con 0x7fc5a4108610 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.437+0000 7fc59b7fe700 1 -- 172.21.15.38:0/375095380 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc5a001ac70 con 0x7fc5a4108610 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 msgr2=0x7fc5a407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 --2- 172.21.15.38:0/375095380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a407db60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00117e0 tx=0x7fc5a0011ce0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 shutdown_connections 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 --2- 172.21.15.38:0/375095380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a4108610 0x7fc5a407db60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 >> 172.21.15.38:0/375095380 conn(0x7fc5a4103c10 msgr2=0x7fc5a4104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 shutdown_connections 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:48.438+0000 7fc5ac421700 1 -- 172.21.15.38:0/375095380 wait complete. 2024-04-25T12:54:49.028 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (1/15)... 2024-04-25T12:54:50.217 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:49 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:49.864+0000 7fe730042200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:54:50.217 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:50.044+0000 7fe730042200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:54:50.217 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:50.219+0000 7fe730042200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:54:50.821 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:50.537+0000 7fe730042200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:54:50.821 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:50.638+0000 7fe730042200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:54:51.149 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:50.850+0000 7fe730042200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:54:51.149 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:51.056+0000 7fe730042200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:54:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:51 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2475238167' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:54:52.070 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:51.716+0000 7fe730042200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:54:52.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:52.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:54:52.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:54:52.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:52.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:54:40.452239+0000", 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.487+0000 7ff08cc1e700 1 Processor -- start 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.487+0000 7ff08cc1e700 1 -- start start 2024-04-25T12:54:52.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.487+0000 7ff08cc1e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff0881089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.487+0000 7ff08cc1e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff088108fb0 con 0x7ff0881085c0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.487+0000 7ff08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff0881089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff0881089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46152/0 (socket says 172.21.15.38:46152) 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08659c700 1 -- 172.21.15.38:0/2298996068 learned_addr learned my addr 172.21.15.38:0/2298996068 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08659c700 1 -- 172.21.15.38:0/2298996068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088109770 con 0x7ff0881085c0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08659c700 1 --2- 172.21.15.38:0/2298996068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff0881089e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0780098a0 tx=0x7ff078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ee45915879eac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08559a700 1 -- 172.21.15.38:0/2298996068 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078004160 con 0x7ff0881085c0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08559a700 1 -- 172.21.15.38:0/2298996068 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7ff078004300 con 0x7ff0881085c0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.488+0000 7ff08559a700 1 -- 172.21.15.38:0/2298996068 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078011b20 con 0x7ff0881085c0 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2298996068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 msgr2=0x7ff0881089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 --2- 172.21.15.38:0/2298996068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff0881089e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0780098a0 tx=0x7ff078009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2298996068 shutdown_connections 2024-04-25T12:54:52.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 --2- 172.21.15.38:0/2298996068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff0881089e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2298996068 >> 172.21.15.38:0/2298996068 conn(0x7ff088103be0 msgr2=0x7ff088106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2298996068 shutdown_connections 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.489+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2298996068 wait complete. 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08cc1e700 1 Processor -- start 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08cc1e700 1 -- start start 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08cc1e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff08819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08cc1e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810cb20 con 0x7ff0881085c0 2024-04-25T12:54:52.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff08819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff08819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46164/0 (socket says 172.21.15.38:46164) 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.490+0000 7ff08659c700 1 -- 172.21.15.38:0/2475238167 learned_addr learned my addr 172.21.15.38:0/2475238167 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff08659c700 1 -- 172.21.15.38:0/2475238167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08819c910 con 0x7ff0881085c0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff08659c700 1 --2- 172.21.15.38:0/2475238167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff08819c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff0780117e0 tx=0x7ff078011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078003b20 con 0x7ff0881085c0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7ff078003cc0 con 0x7ff0881085c0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08819cba0 con 0x7ff0881085c0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078019870 con 0x7ff0881085c0 2024-04-25T12:54:52.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08819d000 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff07802d430 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.491+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0780247d0 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.492+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0880640a0 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.495+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff078011e20 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.569+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff08819fdb0 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.569+0000 7ff0777fe700 1 -- 172.21.15.38:0/2475238167 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff07801ac70 con 0x7ff0881085c0 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.570+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 msgr2=0x7ff08819c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.570+0000 7ff08cc1e700 1 --2- 172.21.15.38:0/2475238167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff08819c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff0780117e0 tx=0x7ff078011ce0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.570+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 shutdown_connections 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.570+0000 7ff08cc1e700 1 --2- 172.21.15.38:0/2475238167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0881085c0 0x7ff08819c3d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:52.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.570+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 >> 172.21.15.38:0/2475238167 conn(0x7ff088103be0 msgr2=0x7ff088104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:52.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.570+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 shutdown_connections 2024-04-25T12:54:52.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:51.571+0000 7ff08cc1e700 1 -- 172.21.15.38:0/2475238167 wait complete. 2024-04-25T12:54:52.205 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (2/15)... 2024-04-25T12:54:52.470 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:52.161+0000 7fe730042200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:54:52.470 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:52.262+0000 7fe730042200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:54:52.470 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:52.472+0000 7fe730042200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:54:53.320 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:53.020+0000 7fe730042200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:54:53.320 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:53.116+0000 7fe730042200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:54:53.721 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:53.367+0000 7fe730042200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:54:53.721 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:53.506+0000 7fe730042200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:54:53.721 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:53.604+0000 7fe730042200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:54:53.721 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:53.723+0000 7fe730042200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:54:54.320 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:54.062+0000 7fe730042200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:54:54.945 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:54 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1309902685' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:54:54.946 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:54.708+0000 7fe730042200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:54:54.946 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:54:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:54:54.948+0000 7fe730042200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:54:55.326 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:55.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:54:55.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:54:55.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T12:54:55.346 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:54:40.452239+0000", 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:54:55.347 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.650+0000 7fc068ae2700 1 Processor -- start 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.650+0000 7fc068ae2700 1 -- start start 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.650+0000 7fc068ae2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc064108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.650+0000 7fc068ae2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc064109000 con 0x7fc064108610 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc064108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc064108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46174/0 (socket says 172.21.15.38:46174) 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06259c700 1 -- 172.21.15.38:0/2141208346 learned_addr learned my addr 172.21.15.38:0/2141208346 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06259c700 1 -- 172.21.15.38:0/2141208346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0641097c0 con 0x7fc064108610 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06259c700 1 --2- 172.21.15.38:0/2141208346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc064108a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098a0 tx=0x7fc054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1547b40457d8ae48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06159a700 1 -- 172.21.15.38:0/2141208346 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054004160 con 0x7fc064108610 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06159a700 1 -- 172.21.15.38:0/2141208346 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc054004300 con 0x7fc064108610 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.651+0000 7fc06159a700 1 -- 172.21.15.38:0/2141208346 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054011b20 con 0x7fc064108610 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 -- 172.21.15.38:0/2141208346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 msgr2=0x7fc064108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:55.348 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 --2- 172.21.15.38:0/2141208346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc064108a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098a0 tx=0x7fc054009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 -- 172.21.15.38:0/2141208346 shutdown_connections 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 --2- 172.21.15.38:0/2141208346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc064108a30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 -- 172.21.15.38:0/2141208346 >> 172.21.15.38:0/2141208346 conn(0x7fc064103c10 msgr2=0x7fc064106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 -- 172.21.15.38:0/2141208346 shutdown_connections 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.652+0000 7fc068ae2700 1 -- 172.21.15.38:0/2141208346 wait complete. 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc068ae2700 1 Processor -- start 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc068ae2700 1 -- start start 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc068ae2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc06419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc068ae2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc06410cb70 con 0x7fc064108610 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc06259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc06419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc06259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc06419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46186/0 (socket says 172.21.15.38:46186) 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc06259c700 1 -- 172.21.15.38:0/1309902685 learned_addr learned my addr 172.21.15.38:0/1309902685 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.653+0000 7fc06259c700 1 -- 172.21.15.38:0/1309902685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06419c900 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc06259c700 1 --2- 172.21.15.38:0/1309902685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc06419c3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0540117e0 tx=0x7fc054011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054003b30 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc054003cd0 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06419cb90 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc05401a830 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06419cff0 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc05402d3f0 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.654+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc054019700 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.655+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0640640a0 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.659+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc054024ba0 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.733+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc06419fda0 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.733+0000 7fc0537fe700 1 -- 172.21.15.38:0/1309902685 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc054032030 con 0x7fc064108610 2024-04-25T12:54:55.350 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 msgr2=0x7fc06419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 --2- 172.21.15.38:0/1309902685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc06419c3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0540117e0 tx=0x7fc054011ce0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 shutdown_connections 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 --2- 172.21.15.38:0/1309902685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc064108610 0x7fc06419c3c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 >> 172.21.15.38:0/1309902685 conn(0x7fc064103c10 msgr2=0x7fc064104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 shutdown_connections 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:54.735+0000 7fc068ae2700 1 -- 172.21.15.38:0/1309902685 wait complete. 2024-04-25T12:54:55.351 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (3/15)... 2024-04-25T12:54:56.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: Activating manager daemon x 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: mgrmap e2: x(active, starting, since 0.00348652s) 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: Manager daemon x is now available 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' 2024-04-25T12:54:56.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:55 smithi038 ceph-mon[29400]: from='mgr.14100 172.21.15.38:0/590373755' entity='mgr.x' 2024-04-25T12:54:57.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:56 smithi038 ceph-mon[29400]: mgrmap e3: x(active, since 1.00789s) 2024-04-25T12:54:58.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:57 smithi038 ceph-mon[29400]: mgrmap e4: x(active, since 2s) 2024-04-25T12:54:58.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:58.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:54:58.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T12:54:58.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T12:54:58.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T12:54:58.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T12:54:58.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T12:54:58.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.698 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T12:54:58.699 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T12:54:58.699 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T12:54:58.699 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T12:54:58.699 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T12:54:58.699 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T12:54:58.699 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T12:54:40.452239+0000", 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabc15a0700 1 Processor -- start 2024-04-25T12:54:58.700 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabc15a0700 1 -- start start 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabc15a0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabc15a0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc07c810 con 0x7fabbc07de20 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46266/0 (socket says 172.21.15.38:46266) 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.736+0000 7fabbbfff700 1 -- 172.21.15.38:0/2296572437 learned_addr learned my addr 172.21.15.38:0/2296572437 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.737+0000 7fabbbfff700 1 -- 172.21.15.38:0/2296572437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc07c990 con 0x7fabbc07de20 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.737+0000 7fabbbfff700 1 --2- 172.21.15.38:0/2296572437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc07c2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098a0 tx=0x7fabac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9e73212eb9364f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.737+0000 7fabbaffd700 1 -- 172.21.15.38:0/2296572437 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac004160 con 0x7fabbc07de20 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.737+0000 7fabbaffd700 1 -- 172.21.15.38:0/2296572437 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fabac004300 con 0x7fabbc07de20 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.737+0000 7fabbaffd700 1 -- 172.21.15.38:0/2296572437 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac011b20 con 0x7fabbc07de20 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 -- 172.21.15.38:0/2296572437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 msgr2=0x7fabbc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 --2- 172.21.15.38:0/2296572437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc07c2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098a0 tx=0x7fabac009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 -- 172.21.15.38:0/2296572437 shutdown_connections 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 --2- 172.21.15.38:0/2296572437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc07c2d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 -- 172.21.15.38:0/2296572437 >> 172.21.15.38:0/2296572437 conn(0x7fabbc103a60 msgr2=0x7fabbc105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 -- 172.21.15.38:0/2296572437 shutdown_connections 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 -- 172.21.15.38:0/2296572437 wait complete. 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 Processor -- start 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.738+0000 7fabc15a0700 1 -- start start 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabc15a0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabc15a0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10e9b0 con 0x7fabbc07de20 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46272/0 (socket says 172.21.15.38:46272) 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabbbfff700 1 -- 172.21.15.38:0/1915467599 learned_addr learned my addr 172.21.15.38:0/1915467599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:58.702 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabbbfff700 1 -- 172.21.15.38:0/1915467599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19e980 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.739+0000 7fabbbfff700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc19e440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fabac0116a0 tx=0x7fabac011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.740+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac003ab0 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.740+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fabac003c50 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.740+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc19ec10 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.740+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac019870 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.740+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc19f070 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.740+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fabac019b30 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.741+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc0640a0 con 0x7fabbc07de20 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.741+0000 7fabb97fa700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7faba40395d0 0x7faba403ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:58.703 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.741+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fabac04de50 con 0x7fabbc07de20 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.741+0000 7fabbb7fe700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7faba40395d0 0x7faba403ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.741+0000 7fabbb7fe700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7faba40395d0 0x7faba403ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006fd0 tx=0x7fabb0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:57.746+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabac011e20 con 0x7fabbc07de20 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.045+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fabbc1a1a90 con 0x7fabbc07de20 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.046+0000 7fabb97fa700 1 -- 172.21.15.38:0/1915467599 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fabbc1a1a90 con 0x7fabbc07de20 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7faba40395d0 msgr2=0x7faba403ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7faba40395d0 0x7faba403ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006fd0 tx=0x7fabb0006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 msgr2=0x7fabbc19e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc19e440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fabac0116a0 tx=0x7fabac011ce0 comp rx=0 tx=0).stop 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 shutdown_connections 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7faba40395d0 0x7faba403ba90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 --2- 172.21.15.38:0/1915467599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabbc07de20 0x7fabbc19e440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 >> 172.21.15.38:0/1915467599 conn(0x7fabbc103a60 msgr2=0x7fabbc1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:58.704 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 shutdown_connections 2024-04-25T12:54:58.705 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:58.048+0000 7fabc15a0700 1 -- 172.21.15.38:0/1915467599 wait complete. 2024-04-25T12:54:58.705 INFO:teuthology.orchestra.run.smithi038.stdout:mgr is available 2024-04-25T12:54:59.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:58 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1915467599' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T12:54:59.971 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T12:54:59.972 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.140+0000 7f71bb473700 1 Processor -- start 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.140+0000 7f71bb473700 1 -- start start 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71bb473700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b4108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71bb473700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4109040 con 0x7f71b4108650 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71b920f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b4108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71b920f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b4108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46274/0 (socket says 172.21.15.38:46274) 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71b920f700 1 -- 172.21.15.38:0/3009544779 learned_addr learned my addr 172.21.15.38:0/3009544779 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71b920f700 1 -- 172.21.15.38:0/3009544779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b4109800 con 0x7f71b4108650 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.141+0000 7f71b920f700 1 --2- 172.21.15.38:0/3009544779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b4108a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71b00098a0 tx=0x7f71b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43e96a0c12f05ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71a7fff700 1 -- 172.21.15.38:0/3009544779 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b0004160 con 0x7f71b4108650 2024-04-25T12:54:59.973 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71a7fff700 1 -- 172.21.15.38:0/3009544779 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f71b0004300 con 0x7f71b4108650 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71a7fff700 1 -- 172.21.15.38:0/3009544779 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b0011b20 con 0x7f71b4108650 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 -- 172.21.15.38:0/3009544779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 msgr2=0x7f71b4108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 --2- 172.21.15.38:0/3009544779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b4108a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71b00098a0 tx=0x7f71b0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 -- 172.21.15.38:0/3009544779 shutdown_connections 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 --2- 172.21.15.38:0/3009544779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b4108a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 -- 172.21.15.38:0/3009544779 >> 172.21.15.38:0/3009544779 conn(0x7f71b4103cb0 msgr2=0x7f71b41060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 -- 172.21.15.38:0/3009544779 shutdown_connections 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.142+0000 7f71bb473700 1 -- 172.21.15.38:0/3009544779 wait complete. 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71bb473700 1 Processor -- start 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71bb473700 1 -- start start 2024-04-25T12:54:59.974 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71bb473700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71bb473700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71b410cbb0 con 0x7f71b4108650 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71b920f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71b920f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46276/0 (socket says 172.21.15.38:46276) 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71b920f700 1 -- 172.21.15.38:0/1805221679 learned_addr learned my addr 172.21.15.38:0/1805221679 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.143+0000 7f71b920f700 1 -- 172.21.15.38:0/1805221679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419c960 con 0x7f71b4108650 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.144+0000 7f71b920f700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b419c420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006af0 tx=0x7f71b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.144+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b0003930 con 0x7f71b4108650 2024-04-25T12:54:59.975 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.144+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f71b0003ad0 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.144+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b419cbf0 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.144+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b0019610 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.144+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b419d050 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.145+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f71b001a430 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.145+0000 7f71a67fc700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f71a0039620 0x7f71a003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.145+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f71b004da00 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.145+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b40640a0 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.145+0000 7f71b8a0e700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f71a0039620 0x7f71a003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.146+0000 7f71b8a0e700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f71a0039620 0x7f71a003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71a8006fd0 tx=0x7f71a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.151+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71b0011e20 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.359+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f71b419f8e0 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.360+0000 7f71a67fc700 1 -- 172.21.15.38:0/1805221679 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f71b00172b0 con 0x7f71b4108650 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f71a0039620 msgr2=0x7f71a003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:59.976 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f71a0039620 0x7f71a003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71a8006fd0 tx=0x7f71a8006e40 comp rx=0 tx=0).stop 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 msgr2=0x7f71b419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b419c420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006af0 tx=0x7f71b0011c10 comp rx=0 tx=0).stop 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 shutdown_connections 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f71a0039620 0x7f71a003bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 --2- 172.21.15.38:0/1805221679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71b4108650 0x7f71b419c420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.363+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 >> 172.21.15.38:0/1805221679 conn(0x7f71b4103cb0 msgr2=0x7f71b41046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.364+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 shutdown_connections 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:54:59.364+0000 7f71bb473700 1 -- 172.21.15.38:0/1805221679 wait complete. 2024-04-25T12:54:59.977 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling cephadm module... 2024-04-25T12:55:00.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:54:59 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1805221679' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T12:55:00.969 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:00 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2960283692' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T12:55:01.259 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setuser ceph since I am not root 2024-04-25T12:55:01.259 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setgroup ceph since I am not root 2024-04-25T12:55:01.259 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:01.142+0000 7f7c7f8c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:55:01.543 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:01.335+0000 7f7c7f8c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:01.543 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:01.440+0000 7f7c7f8c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.439+0000 7f2a48cf5700 1 Processor -- start 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.439+0000 7f2a48cf5700 1 -- start start 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.439+0000 7f2a48cf5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4410b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.439+0000 7f2a48cf5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4407bea0 con 0x7f2a44109010 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4410b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4410b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34924/0 (socket says 172.21.15.38:34924) 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4259c700 1 -- 172.21.15.38:0/615610985 learned_addr learned my addr 172.21.15.38:0/615610985 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4259c700 1 -- 172.21.15.38:0/615610985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4410b940 con 0x7f2a44109010 2024-04-25T12:55:01.552 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4259c700 1 --2- 172.21.15.38:0/615610985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4410b400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098a0 tx=0x7f2a34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615473db59b43a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:01.553 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4159a700 1 -- 172.21.15.38:0/615610985 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a34004160 con 0x7f2a44109010 2024-04-25T12:55:01.553 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4159a700 1 -- 172.21.15.38:0/615610985 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f2a34004300 con 0x7f2a44109010 2024-04-25T12:55:01.553 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.440+0000 7f2a4159a700 1 -- 172.21.15.38:0/615610985 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a34011b20 con 0x7f2a44109010 2024-04-25T12:55:01.553 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 -- 172.21.15.38:0/615610985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 msgr2=0x7f2a4410b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:01.553 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 --2- 172.21.15.38:0/615610985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4410b400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098a0 tx=0x7f2a34009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:01.553 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 -- 172.21.15.38:0/615610985 shutdown_connections 2024-04-25T12:55:01.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 --2- 172.21.15.38:0/615610985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4410b400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:01.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 -- 172.21.15.38:0/615610985 >> 172.21.15.38:0/615610985 conn(0x7f2a44102a40 msgr2=0x7f2a44104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:01.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 -- 172.21.15.38:0/615610985 shutdown_connections 2024-04-25T12:55:01.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.441+0000 7f2a48cf5700 1 -- 172.21.15.38:0/615610985 wait complete. 2024-04-25T12:55:01.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a48cf5700 1 Processor -- start 2024-04-25T12:55:01.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a48cf5700 1 -- start start 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a48cf5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a48cf5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410c970 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34938/0 (socket says 172.21.15.38:34938) 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.442+0000 7f2a4259c700 1 -- 172.21.15.38:0/2960283692 learned_addr learned my addr 172.21.15.38:0/2960283692 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a4259c700 1 -- 172.21.15.38:0/2960283692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4419c940 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a4259c700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4419c400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a340117e0 tx=0x7f2a34011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a34003b30 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f2a34003cd0 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a3401a7c0 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4419cbd0 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.443+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4419d030 con 0x7f2a44109010 2024-04-25T12:55:01.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.444+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2a3401aa70 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.444+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a440640a0 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.444+0000 7f2a337fe700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f2a2c039620 0x7f2a2c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.444+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a3404db30 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.444+0000 7f2a41d9b700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f2a2c039620 0x7f2a2c03bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.445+0000 7f2a41d9b700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f2a2c039620 0x7f2a2c03bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006fd0 tx=0x7f2a38006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.449+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a34019c70 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.720+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2a4419fa50 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.971+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2a4419fa50 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.973+0000 7f2a337fe700 1 -- 172.21.15.38:0/2960283692 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f2a3402d910 con 0x7f2a44109010 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f2a2c039620 msgr2=0x7f2a2c03bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f2a2c039620 0x7f2a2c03bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006fd0 tx=0x7f2a38006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 msgr2=0x7f2a4419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4419c400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a340117e0 tx=0x7f2a34011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:01.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 shutdown_connections 2024-04-25T12:55:01.557 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f2a2c039620 0x7f2a2c03bae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:01.557 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 --2- 172.21.15.38:0/2960283692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a44109010 0x7f2a4419c400 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:01.557 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 >> 172.21.15.38:0/2960283692 conn(0x7f2a44102a40 msgr2=0x7f2a44103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:01.557 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 shutdown_connections 2024-04-25T12:55:01.557 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:00.975+0000 7f2a48cf5700 1 -- 172.21.15.38:0/2960283692 wait complete. 2024-04-25T12:55:01.795 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:01.775+0000 7f7c7f8c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:55:02.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2960283692' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T12:55:02.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-mon[29400]: mgrmap e5: x(active, since 6s) 2024-04-25T12:55:02.071 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:01 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:01.869+0000 7f7c7f8c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:55:02.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:55:02.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T12:55:02.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T12:55:02.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T12:55:02.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.009+0000 7f6ea0823700 1 Processor -- start 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.009+0000 7f6ea0823700 1 -- start start 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.009+0000 7f6ea0823700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.009+0000 7f6ea0823700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c107f00 con 0x7f6e9c107510 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34966/0 (socket says 172.21.15.38:34966) 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9a59c700 1 -- 172.21.15.38:0/4278355649 learned_addr learned my addr 172.21.15.38:0/4278355649 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9a59c700 1 -- 172.21.15.38:0/4278355649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c108710 con 0x7f6e9c107510 2024-04-25T12:55:02.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9a59c700 1 --2- 172.21.15.38:0/4278355649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c107930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e900098a0 tx=0x7f6e90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4446997bdd431b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9959a700 1 -- 172.21.15.38:0/4278355649 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e90004160 con 0x7f6e9c107510 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9959a700 1 -- 172.21.15.38:0/4278355649 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f6e90004300 con 0x7f6e9c107510 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.010+0000 7f6e9959a700 1 -- 172.21.15.38:0/4278355649 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e90011b20 con 0x7f6e9c107510 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 -- 172.21.15.38:0/4278355649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 msgr2=0x7f6e9c107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 --2- 172.21.15.38:0/4278355649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c107930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e900098a0 tx=0x7f6e90009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 -- 172.21.15.38:0/4278355649 shutdown_connections 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 --2- 172.21.15.38:0/4278355649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c107930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 -- 172.21.15.38:0/4278355649 >> 172.21.15.38:0/4278355649 conn(0x7f6e9c102ad0 msgr2=0x7f6e9c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 -- 172.21.15.38:0/4278355649 shutdown_connections 2024-04-25T12:55:02.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.011+0000 7f6ea0823700 1 -- 172.21.15.38:0/4278355649 wait complete. 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6ea0823700 1 Processor -- start 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6ea0823700 1 -- start start 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6ea0823700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6ea0823700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c10bac0 con 0x7f6e9c107510 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6e9a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6e9a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34976/0 (socket says 172.21.15.38:34976) 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6e9a59c700 1 -- 172.21.15.38:0/1121717535 learned_addr learned my addr 172.21.15.38:0/1121717535 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.012+0000 7f6e9a59c700 1 -- 172.21.15.38:0/1121717535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c1053b0 con 0x7f6e9c107510 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.013+0000 7f6e9a59c700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c104e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e900117e0 tx=0x7f6e90011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.013+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e90003b20 con 0x7f6e9c107510 2024-04-25T12:55:02.895 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.013+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f6e90003cc0 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.013+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c1035a0 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.013+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e90019870 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.013+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c103a00 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.014+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9c0640a0 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.014+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6e90019af0 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.014+0000 7f6e8b7fe700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 0x7f6e8403bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.014+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e9004dcf0 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.014+0000 7f6e99d9b700 1 -- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 msgr2=0x7f6e8403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.014+0000 7f6e99d9b700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 0x7f6e8403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.024+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e90011e20 con 0x7f6e9c107510 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.215+0000 7f6e99d9b700 1 -- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 msgr2=0x7f6e8403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.215+0000 7f6e99d9b700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 0x7f6e8403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:55:02.896 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.336+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6e9c195dc0 con 0x7f6e9c107510 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.336+0000 7f6e8b7fe700 1 -- 172.21.15.38:0/1121717535 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6e90029260 con 0x7f6e9c107510 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 msgr2=0x7f6e8403bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 0x7f6e8403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 msgr2=0x7f6e9c104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c104e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e900117e0 tx=0x7f6e90011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 shutdown_connections 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f6e84039620 0x7f6e8403bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 --2- 172.21.15.38:0/1121717535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e9c107510 0x7f6e9c104e70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 >> 172.21.15.38:0/1121717535 conn(0x7f6e9c102ad0 msgr2=0x7f6e9c192fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 shutdown_connections 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:02.340+0000 7f6ea0823700 1 -- 172.21.15.38:0/1121717535 wait complete. 2024-04-25T12:55:02.897 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-04-25T12:55:02.898 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 5... 2024-04-25T12:55:03.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:02 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1121717535' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T12:55:03.276 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:03 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:03.137+0000 7f7c7f8c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:55:03.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:03 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:03.319+0000 7f7c7f8c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:03.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:03 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:03.495+0000 7f7c7f8c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:55:04.070 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:03 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:03.796+0000 7f7c7f8c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:55:04.071 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:03 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:03.892+0000 7f7c7f8c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:55:04.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:04 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:04.101+0000 7f7c7f8c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:55:04.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:04 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:04.301+0000 7f7c7f8c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:05.070 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:04 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:04.927+0000 7f7c7f8c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:55:05.671 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:05 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:05.365+0000 7f7c7f8c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:55:05.671 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:05 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:05.464+0000 7f7c7f8c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:55:05.671 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:05 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:05.673+0000 7f7c7f8c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:55:06.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:06 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:06.239+0000 7f7c7f8c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:55:06.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:06 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:06.336+0000 7f7c7f8c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:55:06.943 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:06 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:06.588+0000 7f7c7f8c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:55:06.943 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:06 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:06.727+0000 7f7c7f8c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:55:06.943 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:06 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:06.825+0000 7f7c7f8c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:55:07.254 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:06 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:06.945+0000 7f7c7f8c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:07.254 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:07 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:07.256+0000 7f7c7f8c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:55:08.159 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:07 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:07.883+0000 7f7c7f8c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:55:08.159 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:08.108+0000 7f7c7f8c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:55:08.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: Active manager daemon x restarted 2024-04-25T12:55:08.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: Activating manager daemon x 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: mgrmap e6: x(active, starting, since 0.00858807s) 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: Manager daemon x is now available 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:08 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:09.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:09 smithi038 ceph-mon[29400]: Found migration_current of "None". Setting to last migration. 2024-04-25T12:55:09.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:09 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:55:09.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:09 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:55:09.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:09 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:09.438 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:09 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:09.438 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:09 smithi038 ceph-mon[29400]: mgrmap e7: x(active, since 1.01012s) 2024-04-25T12:55:09.723 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:55:09.723 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.298+0000 7f4481dc8700 1 Processor -- start 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.298+0000 7f4481dc8700 1 -- start start 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.298+0000 7f4481dc8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c1078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.298+0000 7f4481dc8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f447c107e90 con 0x7f447c1074a0 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c1078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c1078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34990/0 (socket says 172.21.15.38:34990) 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447b7fe700 1 -- 172.21.15.38:0/3542829333 learned_addr learned my addr 172.21.15.38:0/3542829333 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447b7fe700 1 -- 172.21.15.38:0/3542829333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1086a0 con 0x7f447c1074a0 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447b7fe700 1 --2- 172.21.15.38:0/3542829333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c1078c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f447000bb40 tx=0x7f447000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879725cb0a0a244e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447a7fc700 1 -- 172.21.15.38:0/3542829333 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447000dbe0 con 0x7f447c1074a0 2024-04-25T12:55:09.724 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447a7fc700 1 -- 172.21.15.38:0/3542829333 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f44700044e0 con 0x7f447c1074a0 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.299+0000 7f447a7fc700 1 -- 172.21.15.38:0/3542829333 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4470010470 con 0x7f447c1074a0 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 -- 172.21.15.38:0/3542829333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 msgr2=0x7f447c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 --2- 172.21.15.38:0/3542829333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c1078c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f447000bb40 tx=0x7f447000be50 comp rx=0 tx=0).stop 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 -- 172.21.15.38:0/3542829333 shutdown_connections 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 --2- 172.21.15.38:0/3542829333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c1078c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 -- 172.21.15.38:0/3542829333 >> 172.21.15.38:0/3542829333 conn(0x7f447c102a40 msgr2=0x7f447c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 -- 172.21.15.38:0/3542829333 shutdown_connections 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.300+0000 7f4481dc8700 1 -- 172.21.15.38:0/3542829333 wait complete. 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f4481dc8700 1 Processor -- start 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f4481dc8700 1 -- start start 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f4481dc8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f4481dc8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f447c07afc0 con 0x7f447c1074a0 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34992/0 (socket says 172.21.15.38:34992) 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f447b7fe700 1 -- 172.21.15.38:0/2860064080 learned_addr learned my addr 172.21.15.38:0/2860064080 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.301+0000 7f447b7fe700 1 -- 172.21.15.38:0/2860064080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c19dc60 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.302+0000 7f447b7fe700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c19d720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f447000b340 tx=0x7f4470004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.302+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4470010850 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.302+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f447001d070 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.302+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c19def0 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.302+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4470021cc0 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.302+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c19a4b0 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.303+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f447001f030 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.303+0000 7f4478ff9700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.303+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 --> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f446403c240 con 0x7f4464039670 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.303+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f447004d6a0 con 0x7f447c1074a0 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.303+0000 7f447affd700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 msgr2=0x7f446403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.303+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.504+0000 7f447affd700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 msgr2=0x7f446403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.504+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.905+0000 7f447affd700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 msgr2=0x7f446403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:03.905+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:04.706+0000 7f447affd700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 msgr2=0x7f446403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:04.706+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:06.307+0000 7f447affd700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 msgr2=0x7f446403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2310511598 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:06.307+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:08.117+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f4470019910 con 0x7f447c1074a0 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:08.117+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 msgr2=0x7f446403bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:08.117+0000 7f4478ff9700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.117+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f447004f170 con 0x7f447c1074a0 2024-04-25T12:55:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.118+0000 7f4478ff9700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f446403cf60 0x7f446403f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.118+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f446403c240 con 0x7f446403cf60 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.118+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f446403cf60 0x7f446403f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.119+0000 7f447affd700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f446403cf60 0x7f446403f350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f446c003a10 tx=0x7f446c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.119+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f446403c240 con 0x7f446403cf60 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.130+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f447c19e0c0 con 0x7f446403cf60 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4478ff9700 1 -- 172.21.15.38:0/2860064080 <== mgr.14120 v2:172.21.15.38:6800/3157707598 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f447c19e0c0 con 0x7f446403cf60 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f446403cf60 msgr2=0x7f446403f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f446403cf60 0x7f446403f350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f446c003a10 tx=0x7f446c009290 comp rx=0 tx=0).stop 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 msgr2=0x7f447c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c19d720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f447000b340 tx=0x7f4470004080 comp rx=0 tx=0).stop 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 shutdown_connections 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c1074a0 0x7f447c19d720 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f446403cf60 0x7f446403f350 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:09.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 --2- 172.21.15.38:0/2860064080 >> [v2:172.21.15.38:6800/2310511598,v1:172.21.15.38:6801/2310511598] conn(0x7f4464039670 0x7f446403bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:09.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 >> 172.21.15.38:0/2860064080 conn(0x7f447c102a40 msgr2=0x7f447c190880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:09.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 shutdown_connections 2024-04-25T12:55:09.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:09.131+0000 7f4481dc8700 1 -- 172.21.15.38:0/2860064080 wait complete. 2024-04-25T12:55:09.729 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 5 is available 2024-04-25T12:55:09.729 INFO:teuthology.orchestra.run.smithi038.stdout:Setting orchestrator backend to cephadm... 2024-04-25T12:55:10.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T12:55:10.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T12:55:10.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:09] ENGINE Bus STARTING 2024-04-25T12:55:10.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:09] ENGINE Serving on https://172.21.15.38:7150 2024-04-25T12:55:10.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:09] ENGINE Serving on http://172.21.15.38:8765 2024-04-25T12:55:10.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:09] ENGINE Bus STARTED 2024-04-25T12:55:10.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:10 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32edced700 1 Processor -- start 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32edced700 1 -- start start 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32edced700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e810b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32edced700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32e807bea0 con 0x7f32e8109030 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e810b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e810b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52318/0 (socket says 172.21.15.38:52318) 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32e77fe700 1 -- 172.21.15.38:0/592655323 learned_addr learned my addr 172.21.15.38:0/592655323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.166+0000 7f32e77fe700 1 -- 172.21.15.38:0/592655323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e810b960 con 0x7f32e8109030 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32e77fe700 1 --2- 172.21.15.38:0/592655323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e810b420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32d80098a0 tx=0x7f32d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eee4a2e80d7f061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32e67fc700 1 -- 172.21.15.38:0/592655323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d8004160 con 0x7f32e8109030 2024-04-25T12:55:10.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32e67fc700 1 -- 172.21.15.38:0/592655323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f32d8004300 con 0x7f32e8109030 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32e67fc700 1 -- 172.21.15.38:0/592655323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d8011b20 con 0x7f32e8109030 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32edced700 1 -- 172.21.15.38:0/592655323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 msgr2=0x7f32e810b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32edced700 1 --2- 172.21.15.38:0/592655323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e810b420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32d80098a0 tx=0x7f32d8009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32edced700 1 -- 172.21.15.38:0/592655323 shutdown_connections 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32edced700 1 --2- 172.21.15.38:0/592655323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e810b420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.167+0000 7f32edced700 1 -- 172.21.15.38:0/592655323 >> 172.21.15.38:0/592655323 conn(0x7f32e8102a80 msgr2=0x7f32e8104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.168+0000 7f32edced700 1 -- 172.21.15.38:0/592655323 shutdown_connections 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.168+0000 7f32edced700 1 -- 172.21.15.38:0/592655323 wait complete. 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.168+0000 7f32edced700 1 Processor -- start 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.168+0000 7f32edced700 1 -- start start 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.168+0000 7f32edced700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.168+0000 7f32edced700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32e810c990 con 0x7f32e8109030 2024-04-25T12:55:10.998 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.169+0000 7f32e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.169+0000 7f32e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52328/0 (socket says 172.21.15.38:52328) 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.169+0000 7f32e77fe700 1 -- 172.21.15.38:0/1139049307 learned_addr learned my addr 172.21.15.38:0/1139049307 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.169+0000 7f32e77fe700 1 -- 172.21.15.38:0/1139049307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e819c930 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.169+0000 7f32e77fe700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e819c3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32d80117e0 tx=0x7f32d8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.172+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d8003b30 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.172+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f32d8003cd0 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.172+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e819cbc0 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.172+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d801a7c0 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.172+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e819d020 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.173+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f32d801aa70 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.173+0000 7f32e4ff9700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f32d0039670 0x7f32d003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.173+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f32d804ea30 con 0x7f32e8109030 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.173+0000 7f32e6ffd700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f32d0039670 0x7f32d003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:10.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.173+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e80640a0 con 0x7f32e8109030 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.173+0000 7f32e6ffd700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f32d0039670 0x7f32d003bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32dc006fd0 tx=0x7f32dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.178+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32d8019c70 con 0x7f32e8109030 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.404+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f32e8103b30 con 0x7f32d0039670 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.413+0000 7f32e4ff9700 1 -- 172.21.15.38:0/1139049307 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f32e8103b30 con 0x7f32d0039670 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f32d0039670 msgr2=0x7f32d003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f32d0039670 0x7f32d003bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32dc006fd0 tx=0x7f32dc006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 msgr2=0x7f32e819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e819c3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32d80117e0 tx=0x7f32d8011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 shutdown_connections 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f32d0039670 0x7f32d003bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 --2- 172.21.15.38:0/1139049307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32e8109030 0x7f32e819c3f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 >> 172.21.15.38:0/1139049307 conn(0x7f32e8102a80 msgr2=0x7f32e8103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 shutdown_connections 2024-04-25T12:55:11.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:10.416+0000 7f32edced700 1 -- 172.21.15.38:0/1139049307 wait complete. 2024-04-25T12:55:11.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:11 smithi038 ceph-mon[29400]: mgrmap e8: x(active, since 2s) 2024-04-25T12:55:11.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:11 smithi038 ceph-mon[29400]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:11.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:11 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:11.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:11 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:12.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T12:55:12.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd6011ca700 1 Processor -- start 2024-04-25T12:55:12.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd6011ca700 1 -- start start 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd6011ca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc10b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd6011ca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc07be60 con 0x7fd5fc109070 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc10b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc10b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52340/0 (socket says 172.21.15.38:52340) 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd5fad9d700 1 -- 172.21.15.38:0/697490692 learned_addr learned my addr 172.21.15.38:0/697490692 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd5fad9d700 1 -- 172.21.15.38:0/697490692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc10b9a0 con 0x7fd5fc109070 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.430+0000 7fd5fad9d700 1 --2- 172.21.15.38:0/697490692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc10b460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0098a0 tx=0x7fd5ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adb8282373df56e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd5f9d9b700 1 -- 172.21.15.38:0/697490692 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ec004160 con 0x7fd5fc109070 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd5f9d9b700 1 -- 172.21.15.38:0/697490692 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fd5ec004300 con 0x7fd5fc109070 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd5f9d9b700 1 -- 172.21.15.38:0/697490692 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ec011b20 con 0x7fd5fc109070 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 -- 172.21.15.38:0/697490692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 msgr2=0x7fd5fc10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 --2- 172.21.15.38:0/697490692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc10b460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0098a0 tx=0x7fd5ec009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:12.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 -- 172.21.15.38:0/697490692 shutdown_connections 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 --2- 172.21.15.38:0/697490692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc10b460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 -- 172.21.15.38:0/697490692 >> 172.21.15.38:0/697490692 conn(0x7fd5fc102a80 msgr2=0x7fd5fc104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 -- 172.21.15.38:0/697490692 shutdown_connections 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.431+0000 7fd6011ca700 1 -- 172.21.15.38:0/697490692 wait complete. 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.432+0000 7fd6011ca700 1 Processor -- start 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.432+0000 7fd6011ca700 1 -- start start 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.432+0000 7fd6011ca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.432+0000 7fd6011ca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10c9c0 con 0x7fd5fc109070 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52350/0 (socket says 172.21.15.38:52350) 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5fad9d700 1 -- 172.21.15.38:0/68736065 learned_addr learned my addr 172.21.15.38:0/68736065 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5fad9d700 1 -- 172.21.15.38:0/68736065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc19c940 con 0x7fd5fc109070 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5fad9d700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc19c400 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0117e0 tx=0x7fd5ec011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:12.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ec003b30 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fd5ec003cd0 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc19cbd0 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ec01a7c0 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.433+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc19d030 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.434+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd5ec01a960 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.434+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc0640a0 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.434+0000 7fd5e3fff700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fd5e40396c0 0x7fd5e403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.434+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5ec04d9f0 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.435+0000 7fd5fa59c700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fd5e40396c0 0x7fd5e403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.435+0000 7fd5fa59c700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fd5e40396c0 0x7fd5e403bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0006fd0 tx=0x7fd5f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.440+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5ec020240 con 0x7fd5fc109070 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.648+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd5fc103af0 con 0x7fd5e40396c0 2024-04-25T12:55:12.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.648+0000 7fd5e3fff700 1 -- 172.21.15.38:0/68736065 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd5fc103af0 con 0x7fd5e40396c0 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.651+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fd5e40396c0 msgr2=0x7fd5e403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.651+0000 7fd6011ca700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fd5e40396c0 0x7fd5e403bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0006fd0 tx=0x7fd5f0006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.651+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 msgr2=0x7fd5fc19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.651+0000 7fd6011ca700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc19c400 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0117e0 tx=0x7fd5ec011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.652+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 shutdown_connections 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.652+0000 7fd6011ca700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fd5e40396c0 0x7fd5e403bb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.652+0000 7fd6011ca700 1 --2- 172.21.15.38:0/68736065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5fc109070 0x7fd5fc19c400 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.652+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 >> 172.21.15.38:0/68736065 conn(0x7fd5fc102a80 msgr2=0x7fd5fc1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.652+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 shutdown_connections 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:11.652+0000 7fd6011ca700 1 -- 172.21.15.38:0/68736065 wait complete. 2024-04-25T12:55:12.254 INFO:teuthology.orchestra.run.smithi038.stdout:Generating ssh key... 2024-04-25T12:55:12.479 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-mon[29400]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:13.265 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: Generating public/private rsa key pair. 2024-04-25T12:55:13.265 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: Your identification has been saved in /tmp/tmpex0qj352/key. 2024-04-25T12:55:13.265 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: Your public key has been saved in /tmp/tmpex0qj352/key.pub. 2024-04-25T12:55:13.265 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: The key fingerprint is: 2024-04-25T12:55:13.265 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: SHA256:6Qy1Hu6+z+TJY0M1Rd/HcjOONAZ5qn2Z9bwFcvW8vNs ceph-d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: The key's randomart image is: 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: +---[RSA 3072]----+ 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | .... | 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | ....oo| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | . o=.=B| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | . o .* *+*| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | . S o. ==+o| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | * o.. + o+| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | =.. . +| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | . =+. o.| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: | .++*o .E| 2024-04-25T12:55:13.266 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: +----[SHA256]-----+ 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.715+0000 7f5c545d2700 1 Processor -- start 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c545d2700 1 -- start start 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c545d2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c545d2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c109040 con 0x7f5c4c108650 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c5236e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c5236e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52360/0 (socket says 172.21.15.38:52360) 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c5236e700 1 -- 172.21.15.38:0/3748499437 learned_addr learned my addr 172.21.15.38:0/3748499437 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.716+0000 7f5c5236e700 1 -- 172.21.15.38:0/3748499437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c109800 con 0x7f5c4c108650 2024-04-25T12:55:13.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c5236e700 1 --2- 172.21.15.38:0/3748499437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c108a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0098a0 tx=0x7f5c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7386d14ae2139f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c5136c700 1 -- 172.21.15.38:0/3748499437 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c3c004160 con 0x7f5c4c108650 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c5136c700 1 -- 172.21.15.38:0/3748499437 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f5c3c004300 con 0x7f5c4c108650 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c5136c700 1 -- 172.21.15.38:0/3748499437 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c3c011b20 con 0x7f5c4c108650 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c545d2700 1 -- 172.21.15.38:0/3748499437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 msgr2=0x7f5c4c108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c545d2700 1 --2- 172.21.15.38:0/3748499437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c108a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0098a0 tx=0x7f5c3c009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c545d2700 1 -- 172.21.15.38:0/3748499437 shutdown_connections 2024-04-25T12:55:13.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c545d2700 1 --2- 172.21.15.38:0/3748499437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c108a70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c545d2700 1 -- 172.21.15.38:0/3748499437 >> 172.21.15.38:0/3748499437 conn(0x7f5c4c103cb0 msgr2=0x7f5c4c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.717+0000 7f5c545d2700 1 -- 172.21.15.38:0/3748499437 shutdown_connections 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c545d2700 1 -- 172.21.15.38:0/3748499437 wait complete. 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c545d2700 1 Processor -- start 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c545d2700 1 -- start start 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c545d2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c545d2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c10cbb0 con 0x7f5c4c108650 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c5236e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c5236e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52366/0 (socket says 172.21.15.38:52366) 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.718+0000 7f5c5236e700 1 -- 172.21.15.38:0/3502807937 learned_addr learned my addr 172.21.15.38:0/3502807937 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:13.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c5236e700 1 -- 172.21.15.38:0/3502807937 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c19eb90 con 0x7f5c4c108650 2024-04-25T12:55:13.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c5236e700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c19e650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c006af0 tx=0x7f5c3c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:13.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c3c003930 con 0x7f5c4c108650 2024-04-25T12:55:13.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f5c3c003ad0 con 0x7f5c4c108650 2024-04-25T12:55:13.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c19ee20 con 0x7f5c4c108650 2024-04-25T12:55:13.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c3c019610 con 0x7f5c4c108650 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.719+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c19f280 con 0x7f5c4c108650 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.720+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c0640a0 con 0x7f5c4c108650 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.720+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5c3c01aa00 con 0x7f5c4c108650 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.720+0000 7f5c437fe700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5c38039670 0x7f5c3803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.720+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c3c04da80 con 0x7f5c4c108650 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.720+0000 7f5c51b6d700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5c38039670 0x7f5c3803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.721+0000 7f5c51b6d700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5c38039670 0x7f5c3803bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.725+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c3c011e20 con 0x7f5c4c108650 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.934+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5c4c104d60 con 0x7f5c38039670 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.980+0000 7f5c437fe700 1 -- 172.21.15.38:0/3502807937 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c4c104d60 con 0x7f5c38039670 2024-04-25T12:55:13.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.982+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5c38039670 msgr2=0x7f5c3803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.982+0000 7f5c545d2700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5c38039670 0x7f5c3803bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c48006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.982+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 msgr2=0x7f5c4c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.982+0000 7f5c545d2700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c19e650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c006af0 tx=0x7f5c3c011c10 comp rx=0 tx=0).stop 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.983+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 shutdown_connections 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.983+0000 7f5c545d2700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5c38039670 0x7f5c3803bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.983+0000 7f5c545d2700 1 --2- 172.21.15.38:0/3502807937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c4c108650 0x7f5c4c19e650 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.983+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 >> 172.21.15.38:0/3502807937 conn(0x7f5c4c103cb0 msgr2=0x7f5c4c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.983+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 shutdown_connections 2024-04-25T12:55:13.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:12.983+0000 7f5c545d2700 1 -- 172.21.15.38:0/3502807937 wait complete. 2024-04-25T12:55:14.260 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:13 smithi038 ceph-mon[29400]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:14.260 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:13 smithi038 ceph-mon[29400]: Generating ssh key... 2024-04-25T12:55:14.260 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:13 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:14.261 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:13 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd3166700 1 Processor -- start 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd3166700 1 -- start start 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd3166700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc1078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd3166700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc107eb0 con 0x7f3dcc1074c0 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd0f02700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc1078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd0f02700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc1078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52374/0 (socket says 172.21.15.38:52374) 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.042+0000 7f3dd0f02700 1 -- 172.21.15.38:0/2444016826 learned_addr learned my addr 172.21.15.38:0/2444016826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.043+0000 7f3dd0f02700 1 -- 172.21.15.38:0/2444016826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1086c0 con 0x7f3dcc1074c0 2024-04-25T12:55:14.830 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.043+0000 7f3dd0f02700 1 --2- 172.21.15.38:0/2444016826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc1078e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00098a0 tx=0x7f3dc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1441606f532ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.043+0000 7f3dcb7fe700 1 -- 172.21.15.38:0/2444016826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dc0004160 con 0x7f3dcc1074c0 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.043+0000 7f3dcb7fe700 1 -- 172.21.15.38:0/2444016826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3dc0004300 con 0x7f3dcc1074c0 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.043+0000 7f3dcb7fe700 1 -- 172.21.15.38:0/2444016826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dc0011b20 con 0x7f3dcc1074c0 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 -- 172.21.15.38:0/2444016826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 msgr2=0x7f3dcc1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 --2- 172.21.15.38:0/2444016826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc1078e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00098a0 tx=0x7f3dc0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 -- 172.21.15.38:0/2444016826 shutdown_connections 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 --2- 172.21.15.38:0/2444016826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc1078e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 -- 172.21.15.38:0/2444016826 >> 172.21.15.38:0/2444016826 conn(0x7f3dcc102a80 msgr2=0x7f3dcc104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 -- 172.21.15.38:0/2444016826 shutdown_connections 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.044+0000 7f3dd3166700 1 -- 172.21.15.38:0/2444016826 wait complete. 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd3166700 1 Processor -- start 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd3166700 1 -- start start 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd3166700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd3166700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10ba70 con 0x7f3dcc1074c0 2024-04-25T12:55:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd0f02700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd0f02700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52382/0 (socket says 172.21.15.38:52382) 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd0f02700 1 -- 172.21.15.38:0/1257071893 learned_addr learned my addr 172.21.15.38:0/1257071893 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd0f02700 1 -- 172.21.15.38:0/1257071893 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc105350 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dd0f02700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc104e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00117e0 tx=0x7f3dc0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.045+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dc0003b30 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.046+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc103540 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.046+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3dc0003cd0 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.046+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dc001a830 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.046+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc103a00 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.047+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3dc001a9d0 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.047+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0640a0 con 0x7f3dcc1074c0 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.047+0000 7f3dc9ffb700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dbc039670 0x7f3dbc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:14.832 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.047+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3dc004db30 con 0x7f3dcc1074c0 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.047+0000 7f3dcbfff700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dbc039670 0x7f3dbc03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.047+0000 7f3dcbfff700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dbc039670 0x7f3dbc03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.052+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dc0019c70 con 0x7f3dcc1074c0 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.262+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3dcc195d90 con 0x7f3dbc039670 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.263+0000 7f3dc9ffb700 1 -- 172.21.15.38:0/1257071893 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3dcc195d90 con 0x7f3dbc039670 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dbc039670 msgr2=0x7f3dbc03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dbc039670 0x7f3dbc03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db8006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 msgr2=0x7f3dcc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc104e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00117e0 tx=0x7f3dc0011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 shutdown_connections 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dbc039670 0x7f3dbc03bb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 --2- 172.21.15.38:0/1257071893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dcc1074c0 0x7f3dcc104e10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 >> 172.21.15.38:0/1257071893 conn(0x7f3dcc102a80 msgr2=0x7f3dcc192fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:14.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 shutdown_connections 2024-04-25T12:55:14.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:14.265+0000 7f3dd3166700 1 -- 172.21.15.38:0/1257071893 wait complete. 2024-04-25T12:55:14.834 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T12:55:14.834 INFO:teuthology.orchestra.run.smithi038.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T12:55:14.834 INFO:teuthology.orchestra.run.smithi038.stdout:Adding host smithi038... 2024-04-25T12:55:15.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:15 smithi038 ceph-mon[29400]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:16.355 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:16 smithi038 ceph-mon[29400]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:17.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:17 smithi038 ceph-mon[29400]: Deploying cephadm binary to smithi038 2024-04-25T12:55:20.471 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Added host 'smithi038' with addr '172.21.15.38' 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.279+0000 7fa86020c700 1 Processor -- start 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.279+0000 7fa86020c700 1 -- start start 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.279+0000 7fa86020c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa8581089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.279+0000 7fa86020c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa858108fb0 con 0x7fa8581085c0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85dfa8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa8581089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85dfa8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa8581089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52386/0 (socket says 172.21.15.38:52386) 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85dfa8700 1 -- 172.21.15.38:0/489633411 learned_addr learned my addr 172.21.15.38:0/489633411 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85dfa8700 1 -- 172.21.15.38:0/489633411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858109770 con 0x7fa8581085c0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85dfa8700 1 --2- 172.21.15.38:0/489633411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa8581089e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098a0 tx=0x7fa854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5610e6f1887c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85cfa6700 1 -- 172.21.15.38:0/489633411 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa854004160 con 0x7fa8581085c0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85cfa6700 1 -- 172.21.15.38:0/489633411 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa854004300 con 0x7fa8581085c0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.280+0000 7fa85cfa6700 1 -- 172.21.15.38:0/489633411 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa854011b20 con 0x7fa8581085c0 2024-04-25T12:55:20.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 -- 172.21.15.38:0/489633411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 msgr2=0x7fa8581089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 --2- 172.21.15.38:0/489633411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa8581089e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098a0 tx=0x7fa854009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 -- 172.21.15.38:0/489633411 shutdown_connections 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 --2- 172.21.15.38:0/489633411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa8581089e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 -- 172.21.15.38:0/489633411 >> 172.21.15.38:0/489633411 conn(0x7fa858103be0 msgr2=0x7fa858106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 -- 172.21.15.38:0/489633411 shutdown_connections 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 -- 172.21.15.38:0/489633411 wait complete. 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.281+0000 7fa86020c700 1 Processor -- start 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa86020c700 1 -- start start 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa86020c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa85819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa86020c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa85810cb20 con 0x7fa8581085c0 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa85dfa8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa85819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa85dfa8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa85819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52390/0 (socket says 172.21.15.38:52390) 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa85dfa8700 1 -- 172.21.15.38:0/3168956209 learned_addr learned my addr 172.21.15.38:0/3168956209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa85dfa8700 1 -- 172.21.15.38:0/3168956209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85819eb60 con 0x7fa8581085c0 2024-04-25T12:55:20.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa85dfa8700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa85819e620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8540117e0 tx=0x7fa854011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa854003b20 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa854003cc0 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.282+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85819edf0 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.283+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa854019870 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.283+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85819f250 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.283+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa854019aa0 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.284+0000 7fa84effd700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fa844039670 0x7fa84403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.284+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8580640a0 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.284+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa85404d980 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.284+0000 7fa85d7a7700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fa844039670 0x7fa84403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.284+0000 7fa85d7a7700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fa844039670 0x7fa84403bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa848006fd0 tx=0x7fa848006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.289+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa854011e20 con 0x7fa8581085c0 2024-04-25T12:55:20.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:15.506+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}) -- 0x7fa8581a1ba0 con 0x7fa844039670 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.697+0000 7fa84effd700 1 -- 172.21.15.38:0/3168956209 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa8581a1ba0 con 0x7fa844039670 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fa844039670 msgr2=0x7fa84403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fa844039670 0x7fa84403bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa848006fd0 tx=0x7fa848006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 msgr2=0x7fa85819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa85819e620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8540117e0 tx=0x7fa854011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 shutdown_connections 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fa844039670 0x7fa84403bb30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 --2- 172.21.15.38:0/3168956209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8581085c0 0x7fa85819e620 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 >> 172.21.15.38:0/3168956209 conn(0x7fa858103be0 msgr2=0x7fa858104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 shutdown_connections 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:19.700+0000 7fa86020c700 1 -- 172.21.15.38:0/3168956209 wait complete. 2024-04-25T12:55:20.475 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mon service... 2024-04-25T12:55:20.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:20 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:20.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:20 smithi038 ceph-mon[29400]: Added host smithi038 2024-04-25T12:55:20.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:20 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:21.920 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.944+0000 7f47bece5700 1 Processor -- start 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.944+0000 7f47bece5700 1 -- start start 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bece5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bece5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47b807be60 con 0x7f47b81090c0 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bca81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bca81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37878/0 (socket says 172.21.15.38:37878) 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bca81700 1 -- 172.21.15.38:0/2735237827 learned_addr learned my addr 172.21.15.38:0/2735237827 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bca81700 1 -- 172.21.15.38:0/2735237827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b810b9f0 con 0x7f47b81090c0 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.945+0000 7f47bca81700 1 --2- 172.21.15.38:0/2735237827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b810b4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0098a0 tx=0x7f47ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e6d7d60130a1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.946+0000 7f47b77fe700 1 -- 172.21.15.38:0/2735237827 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47ac004160 con 0x7f47b81090c0 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.946+0000 7f47b77fe700 1 -- 172.21.15.38:0/2735237827 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f47ac004300 con 0x7f47b81090c0 2024-04-25T12:55:21.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.946+0000 7f47b77fe700 1 -- 172.21.15.38:0/2735237827 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47ac011b20 con 0x7f47b81090c0 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 -- 172.21.15.38:0/2735237827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 msgr2=0x7f47b810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 --2- 172.21.15.38:0/2735237827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b810b4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0098a0 tx=0x7f47ac009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 -- 172.21.15.38:0/2735237827 shutdown_connections 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 --2- 172.21.15.38:0/2735237827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b810b4b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 -- 172.21.15.38:0/2735237827 >> 172.21.15.38:0/2735237827 conn(0x7f47b8102ad0 msgr2=0x7f47b8104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 -- 172.21.15.38:0/2735237827 shutdown_connections 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.947+0000 7f47bece5700 1 -- 172.21.15.38:0/2735237827 wait complete. 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bece5700 1 Processor -- start 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bece5700 1 -- start start 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bece5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bece5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810c9a0 con 0x7f47b81090c0 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bca81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bca81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37888/0 (socket says 172.21.15.38:37888) 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bca81700 1 -- 172.21.15.38:0/1133777449 learned_addr learned my addr 172.21.15.38:0/1133777449 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bca81700 1 -- 172.21.15.38:0/1133777449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b819c920 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.948+0000 7f47bca81700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b819c3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0117e0 tx=0x7f47ac011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.949+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47ac003b30 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.949+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b819cbb0 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.949+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f47ac003cd0 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.949+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47ac01a7c0 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.949+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b819d010 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.950+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f47ac01aa70 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.950+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b80640a0 con 0x7f47b81090c0 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.950+0000 7f47b5ffb700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f47a8039670 0x7f47a803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.950+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f47ac04daa0 con 0x7f47b81090c0 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.950+0000 7f47b7fff700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f47a8039670 0x7f47a803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.951+0000 7f47b7fff700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f47a8039670 0x7f47a803bb30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f47a4006fd0 tx=0x7f47a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:20.955+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47ac019c70 con 0x7f47b81090c0 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.197+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f47b8103b40 con 0x7f47a8039670 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.202+0000 7f47b5ffb700 1 -- 172.21.15.38:0/1133777449 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f47b8103b40 con 0x7f47a8039670 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f47a8039670 msgr2=0x7f47a803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f47a8039670 0x7f47a803bb30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f47a4006fd0 tx=0x7f47a4006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 msgr2=0x7f47b819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b819c3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0117e0 tx=0x7f47ac011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 shutdown_connections 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f47a8039670 0x7f47a803bb30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 --2- 172.21.15.38:0/1133777449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b81090c0 0x7f47b819c3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:21.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 >> 172.21.15.38:0/1133777449 conn(0x7f47b8102ad0 msgr2=0x7f47b8103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:21.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 shutdown_connections 2024-04-25T12:55:21.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:21.204+0000 7f47bece5700 1 -- 172.21.15.38:0/1133777449 wait complete. 2024-04-25T12:55:21.925 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mgr service... 2024-04-25T12:55:22.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:22 smithi038 ceph-mon[29400]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:22.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:22 smithi038 ceph-mon[29400]: Saving service mon spec with placement count:5 2024-04-25T12:55:22.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:22 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:23.544 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T12:55:23.544 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.483+0000 7f38a5d79700 1 Processor -- start 2024-04-25T12:55:23.544 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.483+0000 7f38a5d79700 1 -- start start 2024-04-25T12:55:23.544 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f38a5d79700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a0107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:23.544 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f38a5d79700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38a007be60 con 0x7f38a0106e40 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a0107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a0107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37892/0 (socket says 172.21.15.38:37892) 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389f7fe700 1 -- 172.21.15.38:0/2157916548 learned_addr learned my addr 172.21.15.38:0/2157916548 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389f7fe700 1 -- 172.21.15.38:0/2157916548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a01077a0 con 0x7f38a0106e40 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389f7fe700 1 --2- 172.21.15.38:0/2157916548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a0107260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38900098a0 tx=0x7f3890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6b6cf1a5ce1434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389e7fc700 1 -- 172.21.15.38:0/2157916548 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3890004160 con 0x7f38a0106e40 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389e7fc700 1 -- 172.21.15.38:0/2157916548 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3890004300 con 0x7f38a0106e40 2024-04-25T12:55:23.549 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.484+0000 7f389e7fc700 1 -- 172.21.15.38:0/2157916548 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3890011b20 con 0x7f38a0106e40 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 -- 172.21.15.38:0/2157916548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 msgr2=0x7f38a0107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 --2- 172.21.15.38:0/2157916548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a0107260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38900098a0 tx=0x7f3890009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 -- 172.21.15.38:0/2157916548 shutdown_connections 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 --2- 172.21.15.38:0/2157916548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a0107260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 -- 172.21.15.38:0/2157916548 >> 172.21.15.38:0/2157916548 conn(0x7f38a0102a40 msgr2=0x7f38a0104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 -- 172.21.15.38:0/2157916548 shutdown_connections 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.485+0000 7f38a5d79700 1 -- 172.21.15.38:0/2157916548 wait complete. 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f38a5d79700 1 Processor -- start 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f38a5d79700 1 -- start start 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f38a5d79700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f38a5d79700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0108830 con 0x7f38a0106e40 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f389f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:23.550 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f389f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37908/0 (socket says 172.21.15.38:37908) 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f389f7fe700 1 -- 172.21.15.38:0/2598187518 learned_addr learned my addr 172.21.15.38:0/2598187518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.486+0000 7f389f7fe700 1 -- 172.21.15.38:0/2598187518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a019eb60 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.487+0000 7f389f7fe700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a019e620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38900117e0 tx=0x7f3890011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.487+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3890003b30 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.487+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a019edf0 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.487+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3890003cd0 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.487+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f389001a7c0 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.487+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a019f250 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.488+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f389001aa70 con 0x7f38a0106e40 2024-04-25T12:55:23.554 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.488+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a00640a0 con 0x7f38a0106e40 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.488+0000 7f389cff9700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3888039670 0x7f388803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.488+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f389004dce0 con 0x7f38a0106e40 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.488+0000 7f389effd700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3888039670 0x7f388803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.489+0000 7f389effd700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3888039670 0x7f388803bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.493+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3890019c70 con 0x7f38a0106e40 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.721+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f38a0062e80 con 0x7f3888039670 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.731+0000 7f389cff9700 1 -- 172.21.15.38:0/2598187518 <== mgr.14120 v2:172.21.15.38:6800/3157707598 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f38a0062e80 con 0x7f3888039670 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3888039670 msgr2=0x7f388803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3888039670 0x7f388803bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 msgr2=0x7f38a019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a019e620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38900117e0 tx=0x7f3890011ce0 comp rx=0 tx=0).stop 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 shutdown_connections 2024-04-25T12:55:23.555 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3888039670 0x7f388803bb30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:23.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 --2- 172.21.15.38:0/2598187518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a0106e40 0x7f38a019e620 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:23.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 >> 172.21.15.38:0/2598187518 conn(0x7f38a0102a40 msgr2=0x7f38a0195300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:23.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 shutdown_connections 2024-04-25T12:55:23.556 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:22.733+0000 7f38a5d79700 1 -- 172.21.15.38:0/2598187518 wait complete. 2024-04-25T12:55:23.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:23 smithi038 ceph-mon[29400]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:23.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:23 smithi038 ceph-mon[29400]: Saving service mgr spec with placement count:2 2024-04-25T12:55:23.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:23 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:25.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:24 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:25.249 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:24 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:25.249 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:24 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/442397716' entity='client.admin' 2024-04-25T12:55:25.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.428+0000 7f401c43f700 1 Processor -- start 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.428+0000 7f401c43f700 1 -- start start 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401c43f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401410b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401c43f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f401407bea0 con 0x7f4014109010 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401a1db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401410b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401a1db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401410b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37912/0 (socket says 172.21.15.38:37912) 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401a1db700 1 -- 172.21.15.38:0/59812166 learned_addr learned my addr 172.21.15.38:0/59812166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401a1db700 1 -- 172.21.15.38:0/59812166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401410b940 con 0x7f4014109010 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.429+0000 7f401a1db700 1 --2- 172.21.15.38:0/59812166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401410b400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40100098a0 tx=0x7f4010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e8e06ec82a19ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f40191d9700 1 -- 172.21.15.38:0/59812166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4010004160 con 0x7f4014109010 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f40191d9700 1 -- 172.21.15.38:0/59812166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4010004300 con 0x7f4014109010 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f40191d9700 1 -- 172.21.15.38:0/59812166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4010011af0 con 0x7f4014109010 2024-04-25T12:55:25.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 -- 172.21.15.38:0/59812166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 msgr2=0x7f401410b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 --2- 172.21.15.38:0/59812166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401410b400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40100098a0 tx=0x7f4010009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 -- 172.21.15.38:0/59812166 shutdown_connections 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 --2- 172.21.15.38:0/59812166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401410b400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 -- 172.21.15.38:0/59812166 >> 172.21.15.38:0/59812166 conn(0x7f4014102a40 msgr2=0x7f4014104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 -- 172.21.15.38:0/59812166 shutdown_connections 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.430+0000 7f401c43f700 1 -- 172.21.15.38:0/59812166 wait complete. 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401c43f700 1 Processor -- start 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401c43f700 1 -- start start 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401c43f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401c43f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f401410c9b0 con 0x7f4014109010 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401a1db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401a1db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37922/0 (socket says 172.21.15.38:37922) 2024-04-25T12:55:25.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.431+0000 7f401a1db700 1 -- 172.21.15.38:0/442397716 learned_addr learned my addr 172.21.15.38:0/442397716 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f401a1db700 1 -- 172.21.15.38:0/442397716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401419c960 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f401a1db700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401419c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40100117b0 tx=0x7f4010003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40100036a0 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f401001a430 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401419cbf0 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4010019530 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.432+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401419f8e0 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.433+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f401001a5e0 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.433+0000 7f400b7fe700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f4000039620 0x7f400003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.433+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40140640a0 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.433+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f401004d800 con 0x7f4014109010 2024-04-25T12:55:25.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.433+0000 7f40199da700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f4000039620 0x7f400003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.434+0000 7f40199da700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f4000039620 0x7f400003bae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4004006fd0 tx=0x7f4004006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.439+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40100172b0 con 0x7f4014109010 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.650+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f401419fe80 con 0x7f4014109010 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.657+0000 7f400b7fe700 1 -- 172.21.15.38:0/442397716 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f401002cc30 con 0x7f4014109010 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f4000039620 msgr2=0x7f400003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f4000039620 0x7f400003bae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4004006fd0 tx=0x7f4004006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 msgr2=0x7f401419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401419c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40100117b0 tx=0x7f4010003eb0 comp rx=0 tx=0).stop 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 shutdown_connections 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f4000039620 0x7f400003bae0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 --2- 172.21.15.38:0/442397716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4014109010 0x7f401419c420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 >> 172.21.15.38:0/442397716 conn(0x7f4014102a40 msgr2=0x7f40141034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 shutdown_connections 2024-04-25T12:55:25.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:24.660+0000 7f401c43f700 1 -- 172.21.15.38:0/442397716 wait complete. 2024-04-25T12:55:27.257 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:26 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3070258986' entity='client.admin' 2024-04-25T12:55:27.258 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:26 smithi038 ceph-mon[29400]: from='mgr.14120 172.21.15.38:0/1181321845' entity='mgr.x' 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f0279e61700 1 Processor -- start 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f0279e61700 1 -- start start 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f0279e61700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f02741078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f0279e61700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0274107eb0 con 0x7f02741074c0 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f02737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f02741078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f02737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f02741078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37932/0 (socket says 172.21.15.38:37932) 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.719+0000 7f02737fe700 1 -- 172.21.15.38:0/44935337 learned_addr learned my addr 172.21.15.38:0/44935337 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.720+0000 7f02737fe700 1 -- 172.21.15.38:0/44935337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02741086c0 con 0x7f02741074c0 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.720+0000 7f02737fe700 1 --2- 172.21.15.38:0/44935337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f02741078e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0264009e80 tx=0x7f026400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571f2986a1433457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:27.267 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.720+0000 7f02727fc700 1 -- 172.21.15.38:0/44935337 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0264017070 con 0x7f02741074c0 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.720+0000 7f02727fc700 1 -- 172.21.15.38:0/44935337 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f026400b8c0 con 0x7f02741074c0 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.720+0000 7f02727fc700 1 -- 172.21.15.38:0/44935337 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0264004500 con 0x7f02741074c0 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 -- 172.21.15.38:0/44935337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 msgr2=0x7f02741078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 --2- 172.21.15.38:0/44935337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f02741078e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0264009e80 tx=0x7f026400b330 comp rx=0 tx=0).stop 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 -- 172.21.15.38:0/44935337 shutdown_connections 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 --2- 172.21.15.38:0/44935337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f02741078e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 -- 172.21.15.38:0/44935337 >> 172.21.15.38:0/44935337 conn(0x7f0274102aa0 msgr2=0x7f0274104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 -- 172.21.15.38:0/44935337 shutdown_connections 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 -- 172.21.15.38:0/44935337 wait complete. 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.721+0000 7f0279e61700 1 Processor -- start 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0279e61700 1 -- start start 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0279e61700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f027419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0279e61700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f027410ba70 con 0x7f02741074c0 2024-04-25T12:55:27.268 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f02737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f027419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f02737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f027419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37946/0 (socket says 172.21.15.38:37946) 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f02737fe700 1 -- 172.21.15.38:0/3070258986 learned_addr learned my addr 172.21.15.38:0/3070258986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f02737fe700 1 -- 172.21.15.38:0/3070258986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027419c9d0 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f02737fe700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f027419c490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0264009e50 tx=0x7f026400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0264021020 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0264003c60 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0264017040 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027419cc60 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.722+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027419f950 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.723+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0264024020 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.723+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02740640a0 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.723+0000 7f0270ff9700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f0260039620 0x7f026003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.724+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f026400f050 con 0x7f02741074c0 2024-04-25T12:55:27.269 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.724+0000 7f0272ffd700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f0260039620 0x7f026003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.724+0000 7f0272ffd700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f0260039620 0x7f026003bae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f025c006fd0 tx=0x7f025c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.729+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0264053020 con 0x7f02741074c0 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.953+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f027402e330 con 0x7f02741074c0 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.956+0000 7f0270ff9700 1 -- 172.21.15.38:0/3070258986 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f026402b020 con 0x7f02741074c0 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f0260039620 msgr2=0x7f026003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f0260039620 0x7f026003bae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f025c006fd0 tx=0x7f025c006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 msgr2=0x7f027419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f027419c490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0264009e50 tx=0x7f026400bd50 comp rx=0 tx=0).stop 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 shutdown_connections 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f0260039620 0x7f026003bae0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 --2- 172.21.15.38:0/3070258986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02741074c0 0x7f027419c490 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 >> 172.21.15.38:0/3070258986 conn(0x7f0274102aa0 msgr2=0x7f0274103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 shutdown_connections 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:25.958+0000 7f0279e61700 1 -- 172.21.15.38:0/3070258986 wait complete. 2024-04-25T12:55:27.270 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling the dashboard module... 2024-04-25T12:55:28.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1129336188' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T12:55:28.277 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setuser ceph since I am not root 2024-04-25T12:55:28.277 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setgroup ceph since I am not root 2024-04-25T12:55:28.278 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:28.207+0000 7f1e1e3ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:55:28.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:28.402+0000 7f1e1e3ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:28.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:28.508+0000 7f1e1e3ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:55:28.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3de2462700 1 Processor -- start 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3de2462700 1 -- start start 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3de2462700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3de2462700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc109000 con 0x7f3ddc108610 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37950/0 (socket says 172.21.15.38:37950) 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3ddbfff700 1 -- 172.21.15.38:0/800024410 learned_addr learned my addr 172.21.15.38:0/800024410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3ddbfff700 1 -- 172.21.15.38:0/800024410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc1097c0 con 0x7f3ddc108610 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.677+0000 7f3ddbfff700 1 --2- 172.21.15.38:0/800024410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00098a0 tx=0x7f3dd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b486ed3f3d4ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.678+0000 7f3ddaffd700 1 -- 172.21.15.38:0/800024410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dd0004160 con 0x7f3ddc108610 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.678+0000 7f3ddaffd700 1 -- 172.21.15.38:0/800024410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3dd0004300 con 0x7f3ddc108610 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.678+0000 7f3ddaffd700 1 -- 172.21.15.38:0/800024410 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dd0011af0 con 0x7f3ddc108610 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.678+0000 7f3de2462700 1 -- 172.21.15.38:0/800024410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 msgr2=0x7f3ddc108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.678+0000 7f3de2462700 1 --2- 172.21.15.38:0/800024410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00098a0 tx=0x7f3dd0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.678+0000 7f3de2462700 1 -- 172.21.15.38:0/800024410 shutdown_connections 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.679+0000 7f3de2462700 1 --2- 172.21.15.38:0/800024410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc108a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.679+0000 7f3de2462700 1 -- 172.21.15.38:0/800024410 >> 172.21.15.38:0/800024410 conn(0x7f3ddc103c10 msgr2=0x7f3ddc106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.679+0000 7f3de2462700 1 -- 172.21.15.38:0/800024410 shutdown_connections 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.679+0000 7f3de2462700 1 -- 172.21.15.38:0/800024410 wait complete. 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.679+0000 7f3de2462700 1 Processor -- start 2024-04-25T12:55:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.679+0000 7f3de2462700 1 -- start start 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3de2462700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3de2462700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc10cb70 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37966/0 (socket says 172.21.15.38:37966) 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3ddbfff700 1 -- 172.21.15.38:0/1129336188 learned_addr learned my addr 172.21.15.38:0/1129336188 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3ddbfff700 1 -- 172.21.15.38:0/1129336188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc07e0a0 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3ddbfff700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc07db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009520 tx=0x7f3dd0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dd00036a0 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc07c290 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3dd001a430 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dd00193c0 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.680+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc07c7d0 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.681+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc0640a0 con 0x7f3ddc108610 2024-04-25T12:55:28.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.682+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3dd0019560 con 0x7f3ddc108610 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.682+0000 7f3dd97fa700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dc4039670 0x7f3dc403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.682+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3dd004d7f0 con 0x7f3ddc108610 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.682+0000 7f3ddb7fe700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dc4039670 0x7f3dc403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.682+0000 7f3ddb7fe700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dc4039670 0x7f3dc403bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc006fd0 tx=0x7f3dcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.691+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dd000f2e0 con 0x7f3ddc108610 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:27.963+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f3ddc07ca50 con 0x7f3ddc108610 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.019+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f3dd000b7e0 con 0x7f3ddc108610 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.022+0000 7f3dd97fa700 1 -- 172.21.15.38:0/1129336188 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f3dd004fcf0 con 0x7f3ddc108610 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.023+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dc4039670 msgr2=0x7f3dc403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.023+0000 7f3de2462700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dc4039670 0x7f3dc403bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc006fd0 tx=0x7f3dcc006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.023+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 msgr2=0x7f3ddc07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.023+0000 7f3de2462700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc07db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009520 tx=0x7f3dd0003eb0 comp rx=0 tx=0).stop 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.024+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 shutdown_connections 2024-04-25T12:55:28.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.024+0000 7f3de2462700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f3dc4039670 0x7f3dc403bb30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:28.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.024+0000 7f3de2462700 1 --2- 172.21.15.38:0/1129336188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ddc108610 0x7f3ddc07db60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:28.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.024+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 >> 172.21.15.38:0/1129336188 conn(0x7f3ddc103c10 msgr2=0x7f3ddc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:28.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.024+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 shutdown_connections 2024-04-25T12:55:28.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:28.024+0000 7f3de2462700 1 -- 172.21.15.38:0/1129336188 wait complete. 2024-04-25T12:55:28.946 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:28.833+0000 7f1e1e3ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:55:28.947 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:28.922+0000 7f1e1e3ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:55:29.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:29 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1129336188' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T12:55:29.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:29 smithi038 ceph-mon[29400]: mgrmap e9: x(active, since 19s) 2024-04-25T12:55:30.062 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:30 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2196607732' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.147+0000 7fcbebc7c700 1 Processor -- start 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.147+0000 7fcbebc7c700 1 -- start start 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.147+0000 7fcbebc7c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe4108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.147+0000 7fcbebc7c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4109000 con 0x7fcbe4108610 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe9a18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe4108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:30.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe9a18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe4108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37984/0 (socket says 172.21.15.38:37984) 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe9a18700 1 -- 172.21.15.38:0/2660003710 learned_addr learned my addr 172.21.15.38:0/2660003710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe9a18700 1 -- 172.21.15.38:0/2660003710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe41097c0 con 0x7fcbe4108610 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe9a18700 1 --2- 172.21.15.38:0/2660003710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe4108a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098a0 tx=0x7fcbe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2810fc9c462d857d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe8a16700 1 -- 172.21.15.38:0/2660003710 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe0004160 con 0x7fcbe4108610 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe8a16700 1 -- 172.21.15.38:0/2660003710 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcbe0004300 con 0x7fcbe4108610 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.148+0000 7fcbe8a16700 1 -- 172.21.15.38:0/2660003710 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe0011af0 con 0x7fcbe4108610 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2660003710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 msgr2=0x7fcbe4108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 --2- 172.21.15.38:0/2660003710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe4108a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098a0 tx=0x7fcbe0009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2660003710 shutdown_connections 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 --2- 172.21.15.38:0/2660003710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe4108a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2660003710 >> 172.21.15.38:0/2660003710 conn(0x7fcbe4103c10 msgr2=0x7fcbe4106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2660003710 shutdown_connections 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.149+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2660003710 wait complete. 2024-04-25T12:55:30.073 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbebc7c700 1 Processor -- start 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbebc7c700 1 -- start start 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbebc7c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbebc7c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe410cb70 con 0x7fcbe4108610 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbe9a18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbe9a18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37986/0 (socket says 172.21.15.38:37986) 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.150+0000 7fcbe9a18700 1 -- 172.21.15.38:0/2196607732 learned_addr learned my addr 172.21.15.38:0/2196607732 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:30.074 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbe9a18700 1 -- 172.21.15.38:0/2196607732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe407e0a0 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbe9a18700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe407db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009520 tx=0x7fcbe0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe00036a0 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcbe001a430 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe00193c0 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe407c290 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.151+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe407c7d0 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.152+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcbe0019670 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.152+0000 7fcbdaffd700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 0x7fcbd003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.152+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe40640a0 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.152+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcbe004d890 con 0x7fcbe4108610 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.152+0000 7fcbe9217700 1 -- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 msgr2=0x7fcbd003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.152+0000 7fcbe9217700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 0x7fcbd003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:30.075 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.162+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbe000bc70 con 0x7fcbe4108610 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.353+0000 7fcbe9217700 1 -- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 msgr2=0x7fcbd003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.353+0000 7fcbe9217700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 0x7fcbd003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.473+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcbe407d0d0 con 0x7fcbe4108610 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.474+0000 7fcbdaffd700 1 -- 172.21.15.38:0/2196607732 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fcbe000fda0 con 0x7fcbe4108610 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 msgr2=0x7fcbd003bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 0x7fcbd003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 msgr2=0x7fcbe407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe407db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009520 tx=0x7fcbe0003eb0 comp rx=0 tx=0).stop 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 shutdown_connections 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7fcbd0039670 0x7fcbd003bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 --2- 172.21.15.38:0/2196607732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe4108610 0x7fcbe407db60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 >> 172.21.15.38:0/2196607732 conn(0x7fcbe4103c10 msgr2=0x7fcbe4104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:30.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 shutdown_connections 2024-04-25T12:55:30.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:29.476+0000 7fcbebc7c700 1 -- 172.21.15.38:0/2196607732 wait complete. 2024-04-25T12:55:30.077 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-04-25T12:55:30.077 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 9... 2024-04-25T12:55:30.314 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:30.185+0000 7f1e1e3ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:55:30.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:30.369+0000 7f1e1e3ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:30.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:30.551+0000 7f1e1e3ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:55:31.070 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:30.854+0000 7f1e1e3ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:55:31.070 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:30.950+0000 7f1e1e3ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:55:31.362 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:31.162+0000 7f1e1e3ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:55:31.820 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:31.363+0000 7f1e1e3ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:55:32.320 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:31.991+0000 7f1e1e3ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:55:32.752 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:32.434+0000 7f1e1e3ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:55:32.752 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:32.540+0000 7f1e1e3ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:55:32.752 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:32.753+0000 7f1e1e3ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:55:33.570 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:33.305+0000 7f1e1e3ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:55:33.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:33.402+0000 7f1e1e3ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:55:34.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:33.657+0000 7f1e1e3ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:55:34.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:33.797+0000 7f1e1e3ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:55:34.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:33.897+0000 7f1e1e3ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:55:34.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:34.018+0000 7f1e1e3ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:55:34.820 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:34.330+0000 7f1e1e3ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:55:35.241 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:34.962+0000 7f1e1e3ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:55:35.241 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T12:55:35.188+0000 7f1e1e3ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:55:35.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: Active manager daemon x restarted 2024-04-25T12:55:35.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: Activating manager daemon x 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: mgrmap e10: x(active, starting, since 0.00726127s) 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: Manager daemon x is now available 2024-04-25T12:55:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:36.537 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:55:36.537 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:55:36.537 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:36 smithi038 ceph-mon[29400]: mgrmap e11: x(active, since 1.00815s) 2024-04-25T12:55:36.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-04-25T12:55:36.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T12:55:36.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T12:55:36.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-04-25T12:55:36.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.497+0000 7f5118eed700 1 Processor -- start 2024-04-25T12:55:36.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.497+0000 7f5118eed700 1 -- start start 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.497+0000 7f5118eed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f5114108ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.497+0000 7f5118eed700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5114109490 con 0x7f5114108aa0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f5114108ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f5114108ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32874/0 (socket says 172.21.15.38:32874) 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511259c700 1 -- 172.21.15.38:0/3099323535 learned_addr learned my addr 172.21.15.38:0/3099323535 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511259c700 1 -- 172.21.15.38:0/3099323535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5114109610 con 0x7f5114108aa0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511259c700 1 --2- 172.21.15.38:0/3099323535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f5114108ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51040098a0 tx=0x7f5104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111b90c80d0e8503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511159a700 1 -- 172.21.15.38:0/3099323535 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5104004160 con 0x7f5114108aa0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511159a700 1 -- 172.21.15.38:0/3099323535 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5104004300 con 0x7f5114108aa0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.498+0000 7f511159a700 1 -- 172.21.15.38:0/3099323535 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5104011af0 con 0x7f5114108aa0 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 -- 172.21.15.38:0/3099323535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 msgr2=0x7f5114108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 --2- 172.21.15.38:0/3099323535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f5114108ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51040098a0 tx=0x7f5104009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 -- 172.21.15.38:0/3099323535 shutdown_connections 2024-04-25T12:55:36.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 --2- 172.21.15.38:0/3099323535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f5114108ec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 -- 172.21.15.38:0/3099323535 >> 172.21.15.38:0/3099323535 conn(0x7f5114102190 msgr2=0x7f51141045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 -- 172.21.15.38:0/3099323535 shutdown_connections 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.499+0000 7f5118eed700 1 -- 172.21.15.38:0/3099323535 wait complete. 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f5118eed700 1 Processor -- start 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f5118eed700 1 -- start start 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f5118eed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f511419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f5118eed700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51141074c0 con 0x7f5114108aa0 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f511259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f511419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f511259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f511419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32884/0 (socket says 172.21.15.38:32884) 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f511259c700 1 -- 172.21.15.38:0/344970039 learned_addr learned my addr 172.21.15.38:0/344970039 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f511259c700 1 -- 172.21.15.38:0/344970039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511419eae0 con 0x7f5114108aa0 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.500+0000 7f511259c700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f511419e5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5104009520 tx=0x7f5104003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.501+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51040036a0 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.501+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511419edd0 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.501+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f510401a430 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.501+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51040193c0 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.501+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51141a1a60 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.502+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5104019670 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.502+0000 7f510b7fe700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.502+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 --> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f510003c1f0 con 0x7f5100039620 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.502+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f510404d580 con 0x7f5114108aa0 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.502+0000 7f5111d9b700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 msgr2=0x7f510003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.502+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.703+0000 7f5111d9b700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 msgr2=0x7f510003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:30.703+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:31.104+0000 7f5111d9b700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 msgr2=0x7f510003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:36.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:31.104+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:31.905+0000 7f5111d9b700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 msgr2=0x7f510003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:31.905+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:33.506+0000 7f5111d9b700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 msgr2=0x7f510003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3157707598 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:33.506+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:35.196+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f5104019aa0 con 0x7f5114108aa0 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:35.196+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 msgr2=0x7f510003bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:35.196+0000 7f510b7fe700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.197+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f510404f430 con 0x7f5114108aa0 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.197+0000 7f510b7fe700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f510003cfb0 0x7f510003f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.197+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f510003c1f0 con 0x7f510003cfb0 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.198+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f510003cfb0 0x7f510003f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.199+0000 7f5111d9b700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f510003cfb0 0x7f510003f3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50fc003a10 tx=0x7f50fc0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.199+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f510003c1f0 con 0x7f510003cfb0 2024-04-25T12:55:36.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.206+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f51140640a0 con 0x7f510003cfb0 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.206+0000 7f510b7fe700 1 -- 172.21.15.38:0/344970039 <== mgr.14152 v2:172.21.15.38:6800/1333793113 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f51140640a0 con 0x7f510003cfb0 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f510003cfb0 msgr2=0x7f510003f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f510003cfb0 0x7f510003f3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50fc003a10 tx=0x7f50fc0092b0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 msgr2=0x7f511419e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f511419e5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5104009520 tx=0x7f5104003eb0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 shutdown_connections 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5114108aa0 0x7f511419e5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f510003cfb0 0x7f510003f3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 --2- 172.21.15.38:0/344970039 >> [v2:172.21.15.38:6800/3157707598,v1:172.21.15.38:6801/3157707598] conn(0x7f5100039620 0x7f510003bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 >> 172.21.15.38:0/344970039 conn(0x7f5114102190 msgr2=0x7f5114194be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 shutdown_connections 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:36.207+0000 7f5118eed700 1 -- 172.21.15.38:0/344970039 wait complete. 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 9 is available 2024-04-25T12:55:36.945 INFO:teuthology.orchestra.run.smithi038.stdout:Generating a dashboard self-signed certificate... 2024-04-25T12:55:37.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:37 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:36] ENGINE Bus STARTING 2024-04-25T12:55:37.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:37 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:36] ENGINE Serving on http://172.21.15.38:8765 2024-04-25T12:55:37.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:37 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:36] ENGINE Serving on https://172.21.15.38:7150 2024-04-25T12:55:37.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:37 smithi038 ceph-mon[29400]: [25/Apr/2024:12:55:36] ENGINE Bus STARTED 2024-04-25T12:55:38.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:38 smithi038 ceph-mon[29400]: mgrmap e12: x(active, since 2s) 2024-04-25T12:55:38.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:38.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:38 smithi038 ceph-mon[29400]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:38.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:38.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.775+0000 7f0f82ed4700 1 Processor -- start 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.775+0000 7f0f82ed4700 1 -- start start 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.775+0000 7f0f82ed4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c1089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.775+0000 7f0f82ed4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c108fa0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f80c70700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c1089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f80c70700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c1089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32956/0 (socket says 172.21.15.38:32956) 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f80c70700 1 -- 172.21.15.38:0/510994283 learned_addr learned my addr 172.21.15.38:0/510994283 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f80c70700 1 -- 172.21.15.38:0/510994283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c109760 con 0x7f0f7c1085b0 2024-04-25T12:55:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f80c70700 1 --2- 172.21.15.38:0/510994283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c1089d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f700098a0 tx=0x7f0f70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9485cdcd6aedcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f7b7fe700 1 -- 172.21.15.38:0/510994283 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f70004160 con 0x7f0f7c1085b0 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f7b7fe700 1 -- 172.21.15.38:0/510994283 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0f70004300 con 0x7f0f7c1085b0 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.776+0000 7f0f7b7fe700 1 -- 172.21.15.38:0/510994283 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f70011af0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 -- 172.21.15.38:0/510994283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 msgr2=0x7f0f7c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 --2- 172.21.15.38:0/510994283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c1089d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f700098a0 tx=0x7f0f70009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 -- 172.21.15.38:0/510994283 shutdown_connections 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 --2- 172.21.15.38:0/510994283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c1089d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 -- 172.21.15.38:0/510994283 >> 172.21.15.38:0/510994283 conn(0x7f0f7c103bb0 msgr2=0x7f0f7c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 -- 172.21.15.38:0/510994283 shutdown_connections 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.777+0000 7f0f82ed4700 1 -- 172.21.15.38:0/510994283 wait complete. 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f82ed4700 1 Processor -- start 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f82ed4700 1 -- start start 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f82ed4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:38.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f82ed4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c10cb10 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f80c70700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f80c70700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32968/0 (socket says 172.21.15.38:32968) 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f80c70700 1 -- 172.21.15.38:0/3717914821 learned_addr learned my addr 172.21.15.38:0/3717914821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.778+0000 7f0f80c70700 1 -- 172.21.15.38:0/3717914821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c19c950 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.779+0000 7f0f80c70700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c19c410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009520 tx=0x7f0f70003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.779+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f700036a0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.779+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0f7001a430 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.779+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c19cbe0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.779+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f700193c0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.779+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7c19f8d0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.780+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0f7001a5e0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.780+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7c0640a0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.780+0000 7f0f79ffb700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0f6c039670 0x7f0f6c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:38.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.780+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f7004d900 con 0x7f0f7c1085b0 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.781+0000 7f0f7bfff700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0f6c039670 0x7f0f6c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.781+0000 7f0f7bfff700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0f6c039670 0x7f0f6c03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:37.790+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f700196d0 con 0x7f0f7c1085b0 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.001+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0f7c104d30 con 0x7f0f6c039670 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.134+0000 7f0f79ffb700 1 -- 172.21.15.38:0/3717914821 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0f7c104d30 con 0x7f0f6c039670 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0f6c039670 msgr2=0x7f0f6c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0f6c039670 0x7f0f6c03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 msgr2=0x7f0f7c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c19c410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009520 tx=0x7f0f70003eb0 comp rx=0 tx=0).stop 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 shutdown_connections 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0f6c039670 0x7f0f6c03bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:38.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 --2- 172.21.15.38:0/3717914821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c1085b0 0x7f0f7c19c410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:38.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 >> 172.21.15.38:0/3717914821 conn(0x7f0f7c103bb0 msgr2=0x7f0f7c104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:38.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 shutdown_connections 2024-04-25T12:55:38.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:38.136+0000 7f0f82ed4700 1 -- 172.21.15.38:0/3717914821 wait complete. 2024-04-25T12:55:38.718 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial admin user... 2024-04-25T12:55:40.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:39 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:40.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:39 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$lhgpxR9/2NPje.E48Q0XMuoA6ezTRBBYqvjE.rBvOT3VWPedci77u", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714049739, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc3909e0700 1 Processor -- start 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc3909e0700 1 -- start start 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc3909e0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c10b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc3909e0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c07be60 con 0x7fc38c109050 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc38a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c10b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc38a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c10b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32982/0 (socket says 172.21.15.38:32982) 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc38a59c700 1 -- 172.21.15.38:0/607957934 learned_addr learned my addr 172.21.15.38:0/607957934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.203+0000 7fc38a59c700 1 -- 172.21.15.38:0/607957934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c10b980 con 0x7fc38c109050 2024-04-25T12:55:40.558 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.204+0000 7fc38a59c700 1 --2- 172.21.15.38:0/607957934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c10b440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0098a0 tx=0x7fc37c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496c78403ae4d469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.204+0000 7fc38959a700 1 -- 172.21.15.38:0/607957934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc37c004160 con 0x7fc38c109050 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.204+0000 7fc38959a700 1 -- 172.21.15.38:0/607957934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc37c004300 con 0x7fc38c109050 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.204+0000 7fc38959a700 1 -- 172.21.15.38:0/607957934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc37c011af0 con 0x7fc38c109050 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.204+0000 7fc3909e0700 1 -- 172.21.15.38:0/607957934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 msgr2=0x7fc38c10b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.204+0000 7fc3909e0700 1 --2- 172.21.15.38:0/607957934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c10b440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0098a0 tx=0x7fc37c009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 -- 172.21.15.38:0/607957934 shutdown_connections 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 --2- 172.21.15.38:0/607957934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c10b440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 -- 172.21.15.38:0/607957934 >> 172.21.15.38:0/607957934 conn(0x7fc38c102a40 msgr2=0x7fc38c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 -- 172.21.15.38:0/607957934 shutdown_connections 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 -- 172.21.15.38:0/607957934 wait complete. 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 Processor -- start 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.205+0000 7fc3909e0700 1 -- start start 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc3909e0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:40.559 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc3909e0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10c9e0 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc38a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc38a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32994/0 (socket says 172.21.15.38:32994) 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc38a59c700 1 -- 172.21.15.38:0/1815679017 learned_addr learned my addr 172.21.15.38:0/1815679017 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc38a59c700 1 -- 172.21.15.38:0/1815679017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c19c960 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc38a59c700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c19c420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc37c003f90 tx=0x7fc37c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc37c0039e0 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.206+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c19cbf0 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.207+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc37c003b80 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.207+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c19f8e0 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.207+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc37c0196e0 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.207+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc37c022020 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.208+0000 7fc3837fe700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc378039720 0x7fc37803bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.208+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc37c04d9d0 con 0x7fc38c109050 2024-04-25T12:55:40.560 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.208+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c0640a0 con 0x7fc38c109050 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.208+0000 7fc389d9b700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc378039720 0x7fc37803bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.208+0000 7fc389d9b700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc378039720 0x7fc37803bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc374006fd0 tx=0x7fc374006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.217+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc37c020e80 con 0x7fc38c109050 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.449+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc38c19fab0 con 0x7fc378039720 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.698+0000 7fc3837fe700 1 -- 172.21.15.38:0/1815679017 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc38c19fab0 con 0x7fc378039720 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc378039720 msgr2=0x7fc37803bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc378039720 0x7fc37803bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc374006fd0 tx=0x7fc374006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 msgr2=0x7fc38c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c19c420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc37c003f90 tx=0x7fc37c011b70 comp rx=0 tx=0).stop 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 shutdown_connections 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc378039720 0x7fc37803bbe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 --2- 172.21.15.38:0/1815679017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc38c109050 0x7fc38c19c420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 >> 172.21.15.38:0/1815679017 conn(0x7fc38c102a40 msgr2=0x7fc38c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 shutdown_connections 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:39.701+0000 7fc3909e0700 1 -- 172.21.15.38:0/1815679017 wait complete. 2024-04-25T12:55:40.561 INFO:teuthology.orchestra.run.smithi038.stdout:Fetching dashboard port number... 2024-04-25T12:55:40.969 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:40 smithi038 ceph-mon[29400]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:42.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2153306825' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T12:55:42.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T12:55:42.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.000+0000 7f935da2e700 1 Processor -- start 2024-04-25T12:55:42.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.000+0000 7f935da2e700 1 -- start start 2024-04-25T12:55:42.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f935da2e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:42.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f935da2e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f935807c8e0 con 0x7f935807de60 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9356ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9356ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60626/0 (socket says 172.21.15.38:60626) 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9356ffd700 1 -- 172.21.15.38:0/3883188166 learned_addr learned my addr 172.21.15.38:0/3883188166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9356ffd700 1 -- 172.21.15.38:0/3883188166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935807ca60 con 0x7f935807de60 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9356ffd700 1 --2- 172.21.15.38:0/3883188166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935807c310 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93480098a0 tx=0x7f9348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7896a265f172c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9355ffb700 1 -- 172.21.15.38:0/3883188166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9348004160 con 0x7f935807de60 2024-04-25T12:55:42.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9355ffb700 1 -- 172.21.15.38:0/3883188166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9348004300 con 0x7f935807de60 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.001+0000 7f9355ffb700 1 -- 172.21.15.38:0/3883188166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9348011af0 con 0x7f935807de60 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 -- 172.21.15.38:0/3883188166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 msgr2=0x7f935807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 --2- 172.21.15.38:0/3883188166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935807c310 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93480098a0 tx=0x7f9348009bb0 comp rx=0 tx=0).stop 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 -- 172.21.15.38:0/3883188166 shutdown_connections 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 --2- 172.21.15.38:0/3883188166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935807c310 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 -- 172.21.15.38:0/3883188166 >> 172.21.15.38:0/3883188166 conn(0x7f9358103cb0 msgr2=0x7f93581060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:42.040 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 -- 172.21.15.38:0/3883188166 shutdown_connections 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.002+0000 7f935da2e700 1 -- 172.21.15.38:0/3883188166 wait complete. 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f935da2e700 1 Processor -- start 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f935da2e700 1 -- start start 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f935da2e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f935da2e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f935810ec30 con 0x7f935807de60 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f9356ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:42.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f9356ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60636/0 (socket says 172.21.15.38:60636) 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f9356ffd700 1 -- 172.21.15.38:0/2153306825 learned_addr learned my addr 172.21.15.38:0/2153306825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.003+0000 7f9356ffd700 1 -- 172.21.15.38:0/2153306825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935819eb40 con 0x7f935807de60 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.004+0000 7f9356ffd700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935819e600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93480117b0 tx=0x7f93480045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.004+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93480036a0 con 0x7f935807de60 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.004+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f934801a430 con 0x7f935807de60 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.004+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935819edd0 con 0x7f935807de60 2024-04-25T12:55:42.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.004+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9348019530 con 0x7f935807de60 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.004+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93581a1ac0 con 0x7f935807de60 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.005+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9348003840 con 0x7f935807de60 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.005+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93580640a0 con 0x7f935807de60 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.005+0000 7f935ca2c700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93440396d0 0x7f934403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.005+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f934804d8d0 con 0x7f935807de60 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.005+0000 7f93567fc700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93440396d0 0x7f934403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:42.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.005+0000 7f93567fc700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93440396d0 0x7f934403bb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9340006fd0 tx=0x7f9340006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.010+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9348023e50 con 0x7f935807de60 2024-04-25T12:55:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.217+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f93581a2070 con 0x7f935807de60 2024-04-25T12:55:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.218+0000 7f935ca2c700 1 -- 172.21.15.38:0/2153306825 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f934801a900 con 0x7f935807de60 2024-04-25T12:55:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93440396d0 msgr2=0x7f934403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93440396d0 0x7f934403bb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9340006fd0 tx=0x7f9340006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:42.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 msgr2=0x7f935819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935819e600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93480117b0 tx=0x7f93480045f0 comp rx=0 tx=0).stop 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 shutdown_connections 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93440396d0 0x7f934403bb90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 --2- 172.21.15.38:0/2153306825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935807de60 0x7f935819e600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 >> 172.21.15.38:0/2153306825 conn(0x7f9358103cb0 msgr2=0x7f9358104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 shutdown_connections 2024-04-25T12:55:42.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:41.220+0000 7f935da2e700 1 -- 172.21.15.38:0/2153306825 wait complete. 2024-04-25T12:55:42.046 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T12:55:42.047 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-04-25T12:55:42.052 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T12:55:42.052 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-04-25T12:55:42.052 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-04-25T12:55:42.052 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T12:55:42.053 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph Dashboard is now available at: 2024-04-25T12:55:42.054 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:42.054 INFO:teuthology.orchestra.run.smithi038.stdout: URL: https://smithi038:8443/ 2024-04-25T12:55:42.054 INFO:teuthology.orchestra.run.smithi038.stdout: User: admin 2024-04-25T12:55:42.054 INFO:teuthology.orchestra.run.smithi038.stdout: Password: fq2eqnk3cz 2024-04-25T12:55:42.054 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:42.055 INFO:teuthology.orchestra.run.smithi038.stdout:Saving cluster configuration to /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config directory 2024-04-25T12:55:42.055 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling autotune for osd_memory_target 2024-04-25T12:55:43.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:42 smithi038 ceph-mon[29400]: mgrmap e13: x(active, since 6s) 2024-04-25T12:55:43.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.497+0000 7f0066f50700 1 Processor -- start 2024-04-25T12:55:43.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.497+0000 7f0066f50700 1 -- start start 2024-04-25T12:55:43.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0066f50700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f00601078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:43.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0066f50700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0060107e90 con 0x7f00601074a0 2024-04-25T12:55:43.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0064cec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f00601078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0064cec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f00601078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60642/0 (socket says 172.21.15.38:60642) 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0064cec700 1 -- 172.21.15.38:0/453545750 learned_addr learned my addr 172.21.15.38:0/453545750 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0064cec700 1 -- 172.21.15.38:0/453545750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601086a0 con 0x7f00601074a0 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.498+0000 7f0064cec700 1 --2- 172.21.15.38:0/453545750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f00601078c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f005400bb40 tx=0x7f005400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6f337e8b80016d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f005f7fe700 1 -- 172.21.15.38:0/453545750 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f005400dbe0 con 0x7f00601074a0 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f005f7fe700 1 -- 172.21.15.38:0/453545750 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0054004500 con 0x7f00601074a0 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f005f7fe700 1 -- 172.21.15.38:0/453545750 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0054010470 con 0x7f00601074a0 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 -- 172.21.15.38:0/453545750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 msgr2=0x7f00601078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 --2- 172.21.15.38:0/453545750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f00601078c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f005400bb40 tx=0x7f005400be50 comp rx=0 tx=0).stop 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 -- 172.21.15.38:0/453545750 shutdown_connections 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 --2- 172.21.15.38:0/453545750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f00601078c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 -- 172.21.15.38:0/453545750 >> 172.21.15.38:0/453545750 conn(0x7f0060102a40 msgr2=0x7f0060104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:43.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 -- 172.21.15.38:0/453545750 shutdown_connections 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.499+0000 7f0066f50700 1 -- 172.21.15.38:0/453545750 wait complete. 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0066f50700 1 Processor -- start 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0066f50700 1 -- start start 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0066f50700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f006019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0066f50700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f006007afc0 con 0x7f00601074a0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0064cec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f006019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0064cec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f006019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60646/0 (socket says 172.21.15.38:60646) 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.500+0000 7f0064cec700 1 -- 172.21.15.38:0/1725143773 learned_addr learned my addr 172.21.15.38:0/1725143773 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f0064cec700 1 -- 172.21.15.38:0/1725143773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006019a7a0 con 0x7f00601074a0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f0064cec700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f006019a260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f005400bfd0 tx=0x7f00540040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0054027070 con 0x7f00601074a0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0054004300 con 0x7f00601074a0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006019aa30 con 0x7f00601074a0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0054022c70 con 0x7f00601074a0 2024-04-25T12:55:43.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.501+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006019d720 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.502+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f005402a070 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.502+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00600640a0 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.502+0000 7f005dffb700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0050039770 0x7f005003bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.502+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f005401f070 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.502+0000 7f005ffff700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0050039770 0x7f005003bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.503+0000 7f005ffff700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0050039770 0x7f005003bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f004c006fd0 tx=0x7f004c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.507+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0054015070 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.714+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f00601937c0 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.714+0000 7f005dffb700 1 -- 172.21.15.38:0/1725143773 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f00601937c0 con 0x7f00601074a0 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0050039770 msgr2=0x7f005003bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0050039770 0x7f005003bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f004c006fd0 tx=0x7f004c006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 msgr2=0x7f006019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f006019a260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f005400bfd0 tx=0x7f00540040b0 comp rx=0 tx=0).stop 2024-04-25T12:55:43.344 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 shutdown_connections 2024-04-25T12:55:43.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0050039770 0x7f005003bc30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:43.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 --2- 172.21.15.38:0/1725143773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00601074a0 0x7f006019a260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:43.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.716+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 >> 172.21.15.38:0/1725143773 conn(0x7f0060102a40 msgr2=0x7f0060190880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:43.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.717+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 shutdown_connections 2024-04-25T12:55:43.345 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:42.717+0000 7f0066f50700 1 -- 172.21.15.38:0/1725143773 wait complete. 2024-04-25T12:55:44.672 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.757+0000 7f2061c25700 1 Processor -- start 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.757+0000 7f2061c25700 1 -- start start 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f2061c25700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c106d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f2061c25700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f205c107350 con 0x7f205c106960 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60656/0 (socket says 172.21.15.38:60656) 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205b7fe700 1 -- 172.21.15.38:0/484280126 learned_addr learned my addr 172.21.15.38:0/484280126 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:44.673 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205b7fe700 1 -- 172.21.15.38:0/484280126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1098f0 con 0x7f205c106960 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205b7fe700 1 --2- 172.21.15.38:0/484280126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c106d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f204c009e80 tx=0x7f204c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af2bd0dc0e98fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205a7fc700 1 -- 172.21.15.38:0/484280126 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f204c017070 con 0x7f205c106960 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205a7fc700 1 -- 172.21.15.38:0/484280126 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f204c00b8c0 con 0x7f205c106960 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.758+0000 7f205a7fc700 1 -- 172.21.15.38:0/484280126 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f204c004500 con 0x7f205c106960 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 -- 172.21.15.38:0/484280126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 msgr2=0x7f205c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 --2- 172.21.15.38:0/484280126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c106d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f204c009e80 tx=0x7f204c00b330 comp rx=0 tx=0).stop 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 -- 172.21.15.38:0/484280126 shutdown_connections 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 --2- 172.21.15.38:0/484280126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c106d80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 -- 172.21.15.38:0/484280126 >> 172.21.15.38:0/484280126 conn(0x7f205c1022d0 msgr2=0x7f205c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 -- 172.21.15.38:0/484280126 shutdown_connections 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.759+0000 7f2061c25700 1 -- 172.21.15.38:0/484280126 wait complete. 2024-04-25T12:55:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f2061c25700 1 Processor -- start 2024-04-25T12:55:44.675 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f2061c25700 1 -- start start 2024-04-25T12:55:44.675 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f2061c25700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:44.675 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f2061c25700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10cca0 con 0x7f205c106960 2024-04-25T12:55:44.675 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:44.675 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60660/0 (socket says 172.21.15.38:60660) 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.760+0000 7f205b7fe700 1 -- 172.21.15.38:0/1848671352 learned_addr learned my addr 172.21.15.38:0/1848671352 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f205b7fe700 1 -- 172.21.15.38:0/1848671352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c19c9c0 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f205b7fe700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c19c480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f204c009e50 tx=0x7f204c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f204c021020 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c19cc50 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f204c003c60 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f204c017040 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.761+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c19f940 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.762+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f204c024020 con 0x7f205c106960 2024-04-25T12:55:44.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.762+0000 7f2058ff9700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2048039710 0x7f204803bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.762+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f204c00f050 con 0x7f205c106960 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.762+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c0640a0 con 0x7f205c106960 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.762+0000 7f205affd700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2048039710 0x7f204803bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.763+0000 7f205affd700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2048039710 0x7f204803bbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2044006fd0 tx=0x7f2044006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:43.768+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f204c02b020 con 0x7f205c106960 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.090+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f205c19fe30 con 0x7f205c106960 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.094+0000 7f2058ff9700 1 -- 172.21.15.38:0/1848671352 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f204c024310 con 0x7f205c106960 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.096+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2048039710 msgr2=0x7f204803bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.096+0000 7f2061c25700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2048039710 0x7f204803bbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2044006fd0 tx=0x7f2044006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:44.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 msgr2=0x7f205c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c19c480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f204c009e50 tx=0x7f204c00bd50 comp rx=0 tx=0).stop 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 shutdown_connections 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2048039710 0x7f204803bbd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 --2- 172.21.15.38:0/1848671352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f205c106960 0x7f205c19c480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 >> 172.21.15.38:0/1848671352 conn(0x7f205c1022d0 msgr2=0x7f205c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 shutdown_connections 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-04-25T12:55:44.097+0000 7f2061c25700 1 -- 172.21.15.38:0/1848671352 wait complete. 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T12:55:44.679 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.679 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T12:55:44.679 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: ceph telemetry on 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:For more information see: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:55:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:Bootstrap complete. 2024-04-25T12:55:44.716 INFO:tasks.cephadm:Fetching config... 2024-04-25T12:55:44.717 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:55:44.717 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T12:55:44.734 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T12:55:44.734 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:55:44.734 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T12:55:44.791 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T12:55:44.791 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:55:44.791 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T12:55:44.858 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T12:55:44.858 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:55:44.859 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T12:55:44.915 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T12:55:44.915 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T12:55:44.991 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:45.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T12:55:45.041 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:45.053 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T12:55:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBFO48MIBGevObYgChN6PDHV9B4uK5oMZWgk+O0Un5bJnL9MXkUvNZp7pqXydFNwF/k4Xbo8uNjnKLv1X5GfU4ZxSeR7YBaMj8YowGp/zOg9XXN1wMY8mAsG9eGcx05l8c4kh3OCc5IQNpNV1KZukBsepnCAlaUp5JQmkFCdiiUsGhb7dQzYuagMsJ4fw/qxRB4eTtaIkJDXc48QSHzByhJCqKX3X2DBlSpxMGomWobR+ZKsRssrsrcwx5HVDp1kmpVVyBlHVFEXIh69D+fjSOZ/Oxsf9/Cu/trVj0XyyRzYgcj5QslRs0Jwi0/k9ZQ6S2sRmuEjuCRHgQ5REGEknEbe5dxsAR7OdC8FOJJtjx6142u2zSCSJF3d2gjAueaJg3rHoWgXpDi0KkW8CWf88TZtLOEc3Cju8bCm4X6McQhGJNCFJep6TPtcD+U+cTTXv689Dx1ijSZxBm8AGjzcZj1yAYp/+yCX2EKStK6MjBNBB1tJ+W8J6ROvALhBnOzgs= ceph-d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:55:45.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T12:55:45.124 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1848671352' entity='client.admin' 2024-04-25T12:55:45.352 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:47.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.626+0000 7f05f61a4700 1 -- 172.21.15.38:0/2353419871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 msgr2=0x7f05f0100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.626+0000 7f05f61a4700 1 --2- 172.21.15.38:0/2353419871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0100b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009910 tx=0x7f05e0009c20 comp rx=0 tx=0).stop 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.626+0000 7f05f61a4700 1 -- 172.21.15.38:0/2353419871 shutdown_connections 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.626+0000 7f05f61a4700 1 --2- 172.21.15.38:0/2353419871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0100b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.626+0000 7f05f61a4700 1 -- 172.21.15.38:0/2353419871 >> 172.21.15.38:0/2353419871 conn(0x7f05f00fc0f0 msgr2=0x7f05f00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.627+0000 7f05f61a4700 1 -- 172.21.15.38:0/2353419871 shutdown_connections 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.627+0000 7f05f61a4700 1 -- 172.21.15.38:0/2353419871 wait complete. 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.627+0000 7f05f61a4700 1 Processor -- start 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.627+0000 7f05f61a4700 1 -- start start 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.627+0000 7f05f61a4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:47.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.627+0000 7f05f61a4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0108530 con 0x7f05f0100770 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60678/0 (socket says 172.21.15.38:60678) 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ef7fe700 1 -- 172.21.15.38:0/711021775 learned_addr learned my addr 172.21.15.38:0/711021775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ef7fe700 1 -- 172.21.15.38:0/711021775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f0103510 con 0x7f05f0100770 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ef7fe700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0102fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009ec0 tx=0x7f05e0004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e0027070 con 0x7f05f0100770 2024-04-25T12:55:47.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f05e0022470 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f0101700 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e000f5f0 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.628+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f0101c40 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.629+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f05e002a070 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.629+0000 7f05ecff9700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f05d8039650 0x7f05d803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.629+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f05e001e030 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.629+0000 7f05eeffd700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f05d8039650 0x7f05d803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.630+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f0108cb0 con 0x7f05f0100770 2024-04-25T12:55:47.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.630+0000 7f05eeffd700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f05d8039650 0x7f05d803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006fd0 tx=0x7f05e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:47.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.635+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e0013070 con 0x7f05f0100770 2024-04-25T12:55:47.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.841+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f05f0190e00 con 0x7f05f0100770 2024-04-25T12:55:47.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.845+0000 7f05ecff9700 1 -- 172.21.15.38:0/711021775 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f05e0035030 con 0x7f05f0100770 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f05d8039650 msgr2=0x7f05d803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f05d8039650 0x7f05d803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006fd0 tx=0x7f05e4006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 msgr2=0x7f05f0102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0102fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009ec0 tx=0x7f05e0004690 comp rx=0 tx=0).stop 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 shutdown_connections 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f05d8039650 0x7f05d803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 --2- 172.21.15.38:0/711021775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05f0100770 0x7f05f0102fd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:47.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 >> 172.21.15.38:0/711021775 conn(0x7f05f00fc0f0 msgr2=0x7f05f00fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 shutdown_connections 2024-04-25T12:55:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:47.847+0000 7f05f61a4700 1 -- 172.21.15.38:0/711021775 wait complete. 2024-04-25T12:55:48.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:48.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:48.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:55:48.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:48.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:48.495 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:48.495 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/711021775' entity='client.admin' 2024-04-25T12:55:48.495 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:48.495 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:48.521 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T12:55:48.521 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T12:55:48.805 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 -- 172.21.15.38:0/4249631837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 msgr2=0x7f6eb8105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 --2- 172.21.15.38:0/4249631837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8105320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009910 tx=0x7f6ea8009c20 comp rx=0 tx=0).stop 2024-04-25T12:55:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 -- 172.21.15.38:0/4249631837 shutdown_connections 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 --2- 172.21.15.38:0/4249631837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8105320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 -- 172.21.15.38:0/4249631837 >> 172.21.15.38:0/4249631837 conn(0x7f6eb80fc940 msgr2=0x7f6eb80fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 -- 172.21.15.38:0/4249631837 shutdown_connections 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 -- 172.21.15.38:0/4249631837 wait complete. 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.284+0000 7f6ebdc38700 1 Processor -- start 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6ebdc38700 1 -- start start 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6ebdc38700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6ebdc38700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb81084c0 con 0x7f6eb8102f30 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48782/0 (socket says 172.21.15.38:48782) 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb77fe700 1 -- 172.21.15.38:0/300304035 learned_addr learned my addr 172.21.15.38:0/300304035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb77fe700 1 -- 172.21.15.38:0/300304035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb8197ff0 con 0x7f6eb8102f30 2024-04-25T12:55:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb77fe700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8197ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009ef0 tx=0x7f6ea8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea8027070 con 0x7f6eb8102f30 2024-04-25T12:55:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6ea8022470 con 0x7f6eb8102f30 2024-04-25T12:55:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.285+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb81982e0 con 0x7f6eb8102f30 2024-04-25T12:55:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.286+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea800f5f0 con 0x7f6eb8102f30 2024-04-25T12:55:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.286+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb819af70 con 0x7f6eb8102f30 2024-04-25T12:55:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.287+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6ea802a070 con 0x7f6eb8102f30 2024-04-25T12:55:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.287+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb8108c40 con 0x7f6eb8102f30 2024-04-25T12:55:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.287+0000 7f6eb4ff9700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6ea00396b0 0x7f6ea003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.287+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ea801e030 con 0x7f6eb8102f30 2024-04-25T12:55:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.287+0000 7f6eb6ffd700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6ea00396b0 0x7f6ea003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.287+0000 7f6eb6ffd700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6ea00396b0 0x7f6ea003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6eac006fd0 tx=0x7f6eac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.292+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ea8013070 con 0x7f6eb8102f30 2024-04-25T12:55:50.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.504+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6eb802e100 con 0x7f6ea00396b0 2024-04-25T12:55:50.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.507+0000 7f6eb4ff9700 1 -- 172.21.15.38:0/300304035 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6eb802e100 con 0x7f6ea00396b0 2024-04-25T12:55:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.509+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6ea00396b0 msgr2=0x7f6ea003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.509+0000 7f6ebdc38700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6ea00396b0 0x7f6ea003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6eac006fd0 tx=0x7f6eac006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.509+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 msgr2=0x7f6eb8197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.509+0000 7f6ebdc38700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8197ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009ef0 tx=0x7f6ea8004690 comp rx=0 tx=0).stop 2024-04-25T12:55:50.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.510+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 shutdown_connections 2024-04-25T12:55:50.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.510+0000 7f6ebdc38700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6ea00396b0 0x7f6ea003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:50.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.510+0000 7f6ebdc38700 1 --2- 172.21.15.38:0/300304035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb8102f30 0x7f6eb8197ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:50.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.510+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 >> 172.21.15.38:0/300304035 conn(0x7f6eb80fc940 msgr2=0x7f6eb80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:50.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.510+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 shutdown_connections 2024-04-25T12:55:50.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:50.510+0000 7f6ebdc38700 1 -- 172.21.15.38:0/300304035 wait complete. 2024-04-25T12:55:51.314 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-04-25T12:55:51.314 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:55:51.314 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-04-25T12:55:51.333 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:55:51.333 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:51.391 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-04-25T12:55:51.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch host add smithi104 2024-04-25T12:55:51.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:51.635 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:52.633 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:52 smithi038 ceph-mon[29400]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.104+0000 7f831089e700 1 -- 172.21.15.38:0/1918271252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 msgr2=0x7f830c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.104+0000 7f831089e700 1 --2- 172.21.15.38:0/1918271252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c101990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009910 tx=0x7f82fc009c20 comp rx=0 tx=0).stop 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.104+0000 7f831089e700 1 -- 172.21.15.38:0/1918271252 shutdown_connections 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.104+0000 7f831089e700 1 --2- 172.21.15.38:0/1918271252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c101990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.104+0000 7f831089e700 1 -- 172.21.15.38:0/1918271252 >> 172.21.15.38:0/1918271252 conn(0x7f830c0fce40 msgr2=0x7f830c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.105+0000 7f831089e700 1 -- 172.21.15.38:0/1918271252 shutdown_connections 2024-04-25T12:55:53.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.105+0000 7f831089e700 1 -- 172.21.15.38:0/1918271252 wait complete. 2024-04-25T12:55:53.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.105+0000 7f831089e700 1 Processor -- start 2024-04-25T12:55:53.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.105+0000 7f831089e700 1 -- start start 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f831089e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f831089e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10b970 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f830a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f830a59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48788/0 (socket says 172.21.15.38:48788) 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f830a59c700 1 -- 172.21.15.38:0/3965232504 learned_addr learned my addr 172.21.15.38:0/3965232504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f830a59c700 1 -- 172.21.15.38:0/3965232504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c06b6b0 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f830a59c700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c196b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005780 tx=0x7f82fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82fc026070 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f82fc004370 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c197070 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82fc003a30 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.106+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c197530 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.107+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f82fc029070 con 0x7f830c1015b0 2024-04-25T12:55:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.107+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c102a90 con 0x7f830c1015b0 2024-04-25T12:55:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.107+0000 7f83037fe700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f82f80396b0 0x7f82f803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.108+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f82fc01e070 con 0x7f830c1015b0 2024-04-25T12:55:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.108+0000 7f8309d9b700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f82f80396b0 0x7f82f803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.108+0000 7f8309d9b700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f82f80396b0 0x7f82f803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f4006fd0 tx=0x7f82f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.113+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82fc013070 con 0x7f830c1015b0 2024-04-25T12:55:53.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:53.325+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7f830c06af10 con 0x7f82f80396b0 2024-04-25T12:55:54.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:54 smithi038 ceph-mon[29400]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:55:54.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:54 smithi038 ceph-mon[29400]: Deploying cephadm binary to smithi104 2024-04-25T12:55:56.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:56 smithi038 ceph-mon[29400]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.544+0000 7f83037fe700 1 -- 172.21.15.38:0/3965232504 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f830c06af10 con 0x7f82f80396b0 2024-04-25T12:55:57.543 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-04-25T12:55:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.547+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f82f80396b0 msgr2=0x7f82f803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.547+0000 7f831089e700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f82f80396b0 0x7f82f803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f4006fd0 tx=0x7f82f4006e40 comp rx=0 tx=0).stop 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.547+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 msgr2=0x7f830c196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.547+0000 7f831089e700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c196b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005780 tx=0x7f82fc0057b0 comp rx=0 tx=0).stop 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.548+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 shutdown_connections 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.548+0000 7f831089e700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f82f80396b0 0x7f82f803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.548+0000 7f831089e700 1 --2- 172.21.15.38:0/3965232504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f830c1015b0 0x7f830c196b30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.548+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 >> 172.21.15.38:0/3965232504 conn(0x7f830c0fce40 msgr2=0x7f830c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.548+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 shutdown_connections 2024-04-25T12:55:57.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:57.548+0000 7f831089e700 1 -- 172.21.15.38:0/3965232504 wait complete. 2024-04-25T12:55:58.172 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T12:55:58.418 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:55:58.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:58 smithi038 ceph-mon[29400]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:55:58.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:58.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:58 smithi038 ceph-mon[29400]: Added host smithi104 2024-04-25T12:55:58.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:55:58.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:59.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:55:59 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:55:59.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/1358438173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 msgr2=0x7f2f981088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:55:59.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 --2- 172.21.15.38:0/1358438173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f981088c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009910 tx=0x7f2f94009c20 comp rx=0 tx=0).stop 2024-04-25T12:55:59.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/1358438173 shutdown_connections 2024-04-25T12:55:59.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 --2- 172.21.15.38:0/1358438173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f981088c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:55:59.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/1358438173 >> 172.21.15.38:0/1358438173 conn(0x7f2f980fdf80 msgr2=0x7f2f981003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:55:59.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/1358438173 shutdown_connections 2024-04-25T12:55:59.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.921+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/1358438173 wait complete. 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9f7b6700 1 Processor -- start 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9f7b6700 1 -- start start 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9f7b6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f98197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9f7b6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9810ca80 con 0x7f2f981084e0 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9d552700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f98197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9d552700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f98197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37770/0 (socket says 172.21.15.38:37770) 2024-04-25T12:55:59.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.922+0000 7f2f9d552700 1 -- 172.21.15.38:0/770428788 learned_addr learned my addr 172.21.15.38:0/770428788 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f9d552700 1 -- 172.21.15.38:0/770428788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f98198470 con 0x7f2f981084e0 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f9d552700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f98197f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2f94005780 tx=0x7f2f940057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f94026070 con 0x7f2f981084e0 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f98198760 con 0x7f2f981084e0 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2f94004370 con 0x7f2f981084e0 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f94003a30 con 0x7f2f981084e0 2024-04-25T12:55:59.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.923+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9819c7b0 con 0x7f2f981084e0 2024-04-25T12:55:59.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.924+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2f94029070 con 0x7f2f981084e0 2024-04-25T12:55:59.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.924+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f98103b90 con 0x7f2f981084e0 2024-04-25T12:55:59.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.924+0000 7f2f8a7fc700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2f840396b0 0x7f2f8403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:55:59.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.924+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f9401e070 con 0x7f2f981084e0 2024-04-25T12:55:59.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.924+0000 7f2f9cd51700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2f840396b0 0x7f2f8403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:55:59.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.925+0000 7f2f9cd51700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2f840396b0 0x7f2f8403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:55:59.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:55:59.929+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f94013070 con 0x7f2f981084e0 2024-04-25T12:56:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.145+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f98199240 con 0x7f2f840396b0 2024-04-25T12:56:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.146+0000 7f2f8a7fc700 1 -- 172.21.15.38:0/770428788 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f2f98199240 con 0x7f2f840396b0 2024-04-25T12:56:00.143 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:56:00.143 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2f840396b0 msgr2=0x7f2f8403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2f840396b0 0x7f2f8403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).stop 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 msgr2=0x7f2f98197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f98197f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2f94005780 tx=0x7f2f940057b0 comp rx=0 tx=0).stop 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 shutdown_connections 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2f840396b0 0x7f2f8403bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 --2- 172.21.15.38:0/770428788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f981084e0 0x7f2f98197f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 >> 172.21.15.38:0/770428788 conn(0x7f2f980fdf80 msgr2=0x7f2f98109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 shutdown_connections 2024-04-25T12:56:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:00.148+0000 7f2f9f7b6700 1 -- 172.21.15.38:0/770428788 wait complete. 2024-04-25T12:56:00.726 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:00 smithi038 ceph-mon[29400]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:00.726 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:00.756 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-04-25T12:56:00.756 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:56:00.756 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-04-25T12:56:00.775 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:56:00.776 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:00.833 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-04-25T12:56:00.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch host add smithi178 2024-04-25T12:56:01.089 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:02.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:01 smithi038 ceph-mon[29400]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:56:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.578+0000 7fc933b12700 1 -- 172.21.15.38:0/3261644878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 msgr2=0x7fc92c101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.578+0000 7fc933b12700 1 --2- 172.21.15.38:0/3261644878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c101820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc928009910 tx=0x7fc928009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 -- 172.21.15.38:0/3261644878 shutdown_connections 2024-04-25T12:56:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 --2- 172.21.15.38:0/3261644878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c101820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 -- 172.21.15.38:0/3261644878 >> 172.21.15.38:0/3261644878 conn(0x7fc92c0fcea0 msgr2=0x7fc92c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 -- 172.21.15.38:0/3261644878 shutdown_connections 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 -- 172.21.15.38:0/3261644878 wait complete. 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 Processor -- start 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 -- start start 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.579+0000 7fc933b12700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c113110 con 0x7fc92c101440 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc9318ae700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc9318ae700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37794/0 (socket says 172.21.15.38:37794) 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc9318ae700 1 -- 172.21.15.38:0/844686734 learned_addr learned my addr 172.21.15.38:0/844686734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc9318ae700 1 -- 172.21.15.38:0/844686734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c19ea50 con 0x7fc92c101440 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc9318ae700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c19e510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9280057a0 tx=0x7fc928005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc928026070 con 0x7fc92c101440 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc928004370 con 0x7fc92c101440 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc928003b90 con 0x7fc92c101440 2024-04-25T12:56:02.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c19ece0 con 0x7fc92c101440 2024-04-25T12:56:02.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.580+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c19f140 con 0x7fc92c101440 2024-04-25T12:56:02.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.581+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc928029070 con 0x7fc92c101440 2024-04-25T12:56:02.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.581+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92c06b190 con 0x7fc92c101440 2024-04-25T12:56:02.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.581+0000 7fc922ffd700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc918039650 0x7fc91803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:02.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.581+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc92801e070 con 0x7fc92c101440 2024-04-25T12:56:02.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.582+0000 7fc9310ad700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc918039650 0x7fc91803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:02.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.582+0000 7fc9310ad700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc918039650 0x7fc91803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006fd0 tx=0x7fc91c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:02.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.587+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc928013070 con 0x7fc92c101440 2024-04-25T12:56:02.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:02.799+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}) -- 0x7fc92c19f8e0 con 0x7fc918039650 2024-04-25T12:56:03.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:02 smithi038 ceph-mon[29400]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:04.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:03 smithi038 ceph-mon[29400]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:05.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:05.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: Deploying cephadm binary to smithi178 2024-04-25T12:56:05.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:05.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:05.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:05.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:04 smithi038 ceph-mon[29400]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T12:56:07.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.042+0000 7fc922ffd700 1 -- 172.21.15.38:0/844686734 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc92c19f8e0 con 0x7fc918039650 2024-04-25T12:56:07.041 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-04-25T12:56:07.041 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:06 smithi038 ceph-mon[29400]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:07.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc918039650 msgr2=0x7fc91803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc918039650 0x7fc91803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006fd0 tx=0x7fc91c006e40 comp rx=0 tx=0).stop 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 msgr2=0x7fc92c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c19e510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9280057a0 tx=0x7fc928005dc0 comp rx=0 tx=0).stop 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 shutdown_connections 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc918039650 0x7fc91803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 --2- 172.21.15.38:0/844686734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c101440 0x7fc92c19e510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 >> 172.21.15.38:0/844686734 conn(0x7fc92c0fcea0 msgr2=0x7fc92c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 shutdown_connections 2024-04-25T12:56:07.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:07.045+0000 7fc933b12700 1 -- 172.21.15.38:0/844686734 wait complete. 2024-04-25T12:56:07.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T12:56:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:08.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:08.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:08.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:56:08.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:08.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:08.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:08.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:08.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: Added host smithi178 2024-04-25T12:56:08.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:08.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:09.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:09 smithi038 ceph-mon[29400]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:09.065 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:09 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:09.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.572+0000 7f593359e700 1 -- 172.21.15.38:0/1404763129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 msgr2=0x7f5934103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.572+0000 7f593359e700 1 --2- 172.21.15.38:0/1404763129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934103830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5924009910 tx=0x7f5924009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 -- 172.21.15.38:0/1404763129 shutdown_connections 2024-04-25T12:56:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 --2- 172.21.15.38:0/1404763129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934103830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 -- 172.21.15.38:0/1404763129 >> 172.21.15.38:0/1404763129 conn(0x7f59340fcce0 msgr2=0x7f59340ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 -- 172.21.15.38:0/1404763129 shutdown_connections 2024-04-25T12:56:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 -- 172.21.15.38:0/1404763129 wait complete. 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 Processor -- start 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.573+0000 7f593359e700 1 -- start start 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f593410c930 con 0x7f5934103450 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37814/0 (socket says 172.21.15.38:37814) 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593259c700 1 -- 172.21.15.38:0/2141177291 learned_addr learned my addr 172.21.15.38:0/2141177291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593259c700 1 -- 172.21.15.38:0/2141177291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59341982c0 con 0x7f5934103450 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593259c700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934197d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59240057a0 tx=0x7f5924005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5924026070 con 0x7f5934103450 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5924004370 con 0x7f5934103450 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5934198550 con 0x7f5934103450 2024-04-25T12:56:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5924003b90 con 0x7f5934103450 2024-04-25T12:56:09.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.574+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341989b0 con 0x7f5934103450 2024-04-25T12:56:09.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.575+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5924029070 con 0x7f5934103450 2024-04-25T12:56:09.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.575+0000 7f59237fe700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f591c0396b0 0x7f591c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:09.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.575+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f592401e070 con 0x7f5934103450 2024-04-25T12:56:09.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.576+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593406afd0 con 0x7f5934103450 2024-04-25T12:56:09.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.576+0000 7f5931d9b700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f591c0396b0 0x7f591c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:09.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.576+0000 7f5931d9b700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f591c0396b0 0x7f591c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5928006fd0 tx=0x7f5928006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:09.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.581+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5924030020 con 0x7f5934103450 2024-04-25T12:56:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.797+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59341990e0 con 0x7f591c0396b0 2024-04-25T12:56:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.798+0000 7f59237fe700 1 -- 172.21.15.38:0/2141177291 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f59341990e0 con 0x7f591c0396b0 2024-04-25T12:56:09.796 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T12:56:09.796 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-04-25T12:56:09.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f591c0396b0 msgr2=0x7f591c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:09.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f591c0396b0 0x7f591c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5928006fd0 tx=0x7f5928006e40 comp rx=0 tx=0).stop 2024-04-25T12:56:09.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 msgr2=0x7f5934197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:09.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934197d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59240057a0 tx=0x7f5924005dc0 comp rx=0 tx=0).stop 2024-04-25T12:56:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 shutdown_connections 2024-04-25T12:56:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f591c0396b0 0x7f591c03bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 --2- 172.21.15.38:0/2141177291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5934103450 0x7f5934197d80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 >> 172.21.15.38:0/2141177291 conn(0x7f59340fcce0 msgr2=0x7f59341041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 shutdown_connections 2024-04-25T12:56:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:09.800+0000 7f593359e700 1 -- 172.21.15.38:0/2141177291 wait complete. 2024-04-25T12:56:10.433 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T12:56:10.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T12:56:10.676 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:11.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:10 smithi038 ceph-mon[29400]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:11.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:11.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:10 smithi038 ceph-mon[29400]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 -- 172.21.15.38:0/1740900278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 msgr2=0x7f75801019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 --2- 172.21.15.38:0/1740900278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f75801019d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f757c009910 tx=0x7f757c009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 -- 172.21.15.38:0/1740900278 shutdown_connections 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 --2- 172.21.15.38:0/1740900278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f75801019d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 -- 172.21.15.38:0/1740900278 >> 172.21.15.38:0/1740900278 conn(0x7f7580076ea0 msgr2=0x7f75800772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 -- 172.21.15.38:0/1740900278 shutdown_connections 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.148+0000 7f75873e2700 1 -- 172.21.15.38:0/1740900278 wait complete. 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.149+0000 7f75873e2700 1 Processor -- start 2024-04-25T12:56:12.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.149+0000 7f75873e2700 1 -- start start 2024-04-25T12:56:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75873e2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f7580198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75873e2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f758010b960 con 0x7f75801015f0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f758517e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f7580198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f758517e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f7580198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60216/0 (socket says 172.21.15.38:60216) 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f758517e700 1 -- 172.21.15.38:0/1124280651 learned_addr learned my addr 172.21.15.38:0/1124280651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f758517e700 1 -- 172.21.15.38:0/1124280651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580199280 con 0x7f75801015f0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f758517e700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f7580198d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f757c005780 tx=0x7f757c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f757c026070 con 0x7f75801015f0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f757c004370 con 0x7f75801015f0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7580199570 con 0x7f75801015f0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f757c003a30 con 0x7f75801015f0 2024-04-25T12:56:12.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.150+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7580199a30 con 0x7f75801015f0 2024-04-25T12:56:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.151+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f757c029070 con 0x7f75801015f0 2024-04-25T12:56:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.151+0000 7f75767fc700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f756c0396b0 0x7f756c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.151+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7580102a80 con 0x7f75801015f0 2024-04-25T12:56:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.152+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f757c01e070 con 0x7f75801015f0 2024-04-25T12:56:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.152+0000 7f758497d700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f756c0396b0 0x7f756c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.152+0000 7f758497d700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f756c0396b0 0x7f756c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7570006fd0 tx=0x7f7570006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:12.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.157+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f757c030020 con 0x7f75801015f0 2024-04-25T12:56:12.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.369+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f75800640a0 con 0x7f75801015f0 2024-04-25T12:56:12.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.721+0000 7f75767fc700 1 -- 172.21.15.38:0/1124280651 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f757c013070 con 0x7f75801015f0 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f756c0396b0 msgr2=0x7f756c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f756c0396b0 0x7f756c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7570006fd0 tx=0x7f7570006e40 comp rx=0 tx=0).stop 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 msgr2=0x7f7580198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f7580198d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f757c005780 tx=0x7f757c0057b0 comp rx=0 tx=0).stop 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 shutdown_connections 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f756c0396b0 0x7f756c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 --2- 172.21.15.38:0/1124280651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75801015f0 0x7f7580198d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 >> 172.21.15.38:0/1124280651 conn(0x7f7580076ea0 msgr2=0x7f7580102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 shutdown_connections 2024-04-25T12:56:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:12.723+0000 7f75873e2700 1 -- 172.21.15.38:0/1124280651 wait complete. 2024-04-25T12:56:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-04-25T12:56:13.040 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:12 smithi038 ceph-mon[29400]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:13.040 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:12 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1124280651' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T12:56:13.362 INFO:tasks.cephadm:Adding mon.a on smithi038 2024-04-25T12:56:13.362 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-04-25T12:56:13.362 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-04-25T12:56:13.363 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c' 2024-04-25T12:56:13.600 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:14.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1124280651' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T12:56:14.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:13 smithi038 ceph-mon[29400]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T12:56:14.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 -- 172.21.15.178:0/4254718581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 msgr2=0x7f01c81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 --2- 172.21.15.178:0/4254718581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c81019d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009910 tx=0x7f01b8009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 -- 172.21.15.178:0/4254718581 shutdown_connections 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 --2- 172.21.15.178:0/4254718581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c81019d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 -- 172.21.15.178:0/4254718581 >> 172.21.15.178:0/4254718581 conn(0x7f01c8076ea0 msgr2=0x7f01c80772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 -- 172.21.15.178:0/4254718581 shutdown_connections 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.917+0000 7f01cd99b700 1 -- 172.21.15.178:0/4254718581 wait complete. 2024-04-25T12:56:14.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.918+0000 7f01cd99b700 1 Processor -- start 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.918+0000 7f01cd99b700 1 -- start start 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.918+0000 7f01cd99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c8103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.918+0000 7f01cd99b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f01c810b960 con 0x7f01c81015f0 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.918+0000 7f01c6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c8103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01c6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c8103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.178:57116/0 (socket says 172.21.15.178:57116) 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01c6ffd700 1 -- 172.21.15.178:0/431337124 learned_addr learned my addr 172.21.15.178:0/431337124 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01c6ffd700 1 -- 172.21.15.178:0/431337124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c8104f30 con 0x7f01c81015f0 2024-04-25T12:56:14.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01c6ffd700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c8103040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005780 tx=0x7f01b80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b8026070 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f01b8004370 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c81035e0 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b8003af0 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.919+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c8103aa0 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.920+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8102a80 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.920+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f01b8029070 con 0x7f01c81015f0 2024-04-25T12:56:14.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.920+0000 7f01cc999700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f01b0039650 0x7f01b003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:14.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.920+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f01b801e070 con 0x7f01c81015f0 2024-04-25T12:56:14.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.921+0000 7f01c67fc700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f01b0039650 0x7f01b003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:14.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.921+0000 7f01c67fc700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f01b0039650 0x7f01b003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:14.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:14.926+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01b8013070 con 0x7f01c81015f0 2024-04-25T12:56:15.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:15.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:56:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:15.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.139+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}) -- 0x7f01c8103d30 con 0x7f01b0039650 2024-04-25T12:56:15.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.145+0000 7f01cc999700 1 -- 172.21.15.178:0/431337124 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f01c8103d30 con 0x7f01b0039650 2024-04-25T12:56:15.144 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f01b0039650 msgr2=0x7f01b003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f01b0039650 0x7f01b003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).stop 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 msgr2=0x7f01c8103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c8103040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005780 tx=0x7f01b80057b0 comp rx=0 tx=0).stop 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 shutdown_connections 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f01b0039650 0x7f01b003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 --2- 172.21.15.178:0/431337124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01c81015f0 0x7f01c8103040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 >> 172.21.15.178:0/431337124 conn(0x7f01c8076ea0 msgr2=0x7f01c8069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 shutdown_connections 2024-04-25T12:56:15.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:15.148+0000 7f01cd99b700 1 -- 172.21.15.178:0/431337124 wait complete. 2024-04-25T12:56:15.767 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T12:56:15.770 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T12:56:15.772 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T12:56:15.772 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T12:56:16.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c;count:3 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: Deploying daemon mon.c on smithi178 2024-04-25T12:56:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:16 smithi038 ceph-mon[29400]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:18.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:18 smithi038 ceph-mon[29400]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:19.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.798+0000 7f7c8bd57700 1 -- 172.21.15.178:0/2736923181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 msgr2=0x7f7c84104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:19.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.798+0000 7f7c8bd57700 1 --2- 172.21.15.178:0/2736923181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84104770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009910 tx=0x7f7c80009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:19.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.798+0000 7f7c8bd57700 1 -- 172.21.15.178:0/2736923181 shutdown_connections 2024-04-25T12:56:19.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.798+0000 7f7c8bd57700 1 --2- 172.21.15.178:0/2736923181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84104770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.798+0000 7f7c8bd57700 1 -- 172.21.15.178:0/2736923181 >> 172.21.15.178:0/2736923181 conn(0x7f7c840fc5b0 msgr2=0x7f7c840fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.799+0000 7f7c8bd57700 1 -- 172.21.15.178:0/2736923181 shutdown_connections 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.799+0000 7f7c8bd57700 1 -- 172.21.15.178:0/2736923181 wait complete. 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.799+0000 7f7c8bd57700 1 Processor -- start 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.799+0000 7f7c8bd57700 1 -- start start 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.799+0000 7f7c8bd57700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.799+0000 7f7c8bd57700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410cbe0 con 0x7f7c84104390 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c89af3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c89af3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.178:55072/0 (socket says 172.21.15.178:55072) 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c89af3700 1 -- 172.21.15.178:0/3058200832 learned_addr learned my addr 172.21.15.178:0/3058200832 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c89af3700 1 -- 172.21.15.178:0/3058200832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c841984f0 con 0x7f7c84104390 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c89af3700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84197fb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005780 tx=0x7f7c800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c80026070 con 0x7f7c84104390 2024-04-25T12:56:19.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c84198780 con 0x7f7c84104390 2024-04-25T12:56:19.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7c80004370 con 0x7f7c84104390 2024-04-25T12:56:19.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c80003af0 con 0x7f7c84104390 2024-04-25T12:56:19.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.800+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8419c880 con 0x7f7c84104390 2024-04-25T12:56:19.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.801+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c84103bd0 con 0x7f7c84104390 2024-04-25T12:56:19.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.801+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7c80029070 con 0x7f7c84104390 2024-04-25T12:56:19.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.801+0000 7f7c7affd700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7c70039650 0x7f7c7003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:19.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.801+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7c8001e070 con 0x7f7c84104390 2024-04-25T12:56:19.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.802+0000 7f7c892f2700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7c70039650 0x7f7c7003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:19.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.802+0000 7f7c892f2700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7c70039650 0x7f7c7003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7c74006fd0 tx=0x7f7c74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:19.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.807+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c80013070 con 0x7f7c84104390 2024-04-25T12:56:19.871 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 podman[33187]: 2024-04-25 12:56:19.873961606 +0000 UTC m=+0.694840207 container init fd042fa8a8aaa8fe48546f7af2ab2f0d5363d8c8de3f287b47fc3897f8b7470e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T12:56:19.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:19.941+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7c80029360 con 0x7f7c84104390 2024-04-25T12:56:20.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:20.105+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7c840640a0 con 0x7f7c84104390 2024-04-25T12:56:20.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 podman[33187]: 2024-04-25 12:56:19.878457325 +0000 UTC m=+0.699335925 container start fd042fa8a8aaa8fe48546f7af2ab2f0d5363d8c8de3f287b47fc3897f8b7470e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:56:20.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pidfile_write: ignore empty --pid-file 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: load: jerasure load: lrc 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: RocksDB version: 7.9.2 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Git sha 0 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: DB SUMMARY 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: DB Session ID: MQ7PATNP06XO4TVHF5QL 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: CURRENT file: CURRENT 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T12:56:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.error_if_exists: 0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.create_if_missing: 0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.paranoid_checks: 1 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.env: 0x557271d1dcc0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.info_log: 0x557273b05be0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.statistics: (nil) 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.use_fsync: 0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_log_file_size: 0 2024-04-25T12:56:20.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.allow_fallocate: 1 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.use_direct_reads: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.db_log_dir: 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.wal_dir: 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T12:56:20.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.write_buffer_manager: 0x557273a990e0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T12:56:20.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.unordered_write: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.row_cache: None 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.wal_filter: None 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.two_write_queues: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.wal_compression: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.atomic_flush: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T12:56:20.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.log_readahead_size: 0 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_background_jobs: 2 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_background_compactions: -1 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_subcompactions: 1 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T12:56:20.245 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_open_files: -1 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_background_flushes: -1 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Compression algorithms supported: 2024-04-25T12:56:20.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kZSTD supported: 0 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kXpressCompression supported: 0 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kZlibCompression supported: 1 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kSnappyCompression supported: 1 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kLZ4Compression supported: 1 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: kBZip2Compression supported: 0 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.merge_operator: 2024-04-25T12:56:20.247 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_filter: None 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557273b05dc0) 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-04-25T12:56:20.248 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_type: 0 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_index_type: 0 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_shortening: 1 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: checksum: 4 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: no_block_cache: 0 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache: 0x557273acb1f0 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_options: 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: capacity : 536870912 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_shard_bits : 4 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: strict_capacity_limit : 0 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-04-25T12:56:20.249 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_compressed: (nil) 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: persistent_cache: (nil) 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size: 4096 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size_deviation: 10 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_restart_interval: 16 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_block_restart_interval: 1 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: metadata_block_size: 4096 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: partition_filters: 0 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: use_delta_encoding: 1 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: filter_policy: bloomfilter 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: whole_key_filtering: 1 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: verify_compression: 0 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: format_version: 5 2024-04-25T12:56:20.250 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_index_compression: 1 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_align: 0 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout: max_auto_readahead_size: 262144 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout: prepopulate_block_cache: 0 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout: initial_auto_readahead_size: 8192 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression: NoCompression 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.num_levels: 7 2024-04-25T12:56:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T12:56:20.253 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T12:56:20.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T12:56:20.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T12:56:20.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T12:56:20.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T12:56:20.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T12:56:20.256 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T12:56:20.257 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.inplace_update_support: 0 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.bloom_locality: 0 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.max_successive_merges: 0 2024-04-25T12:56:20.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.ttl: 2592000 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.enable_blob_files: false 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.min_blob_size: 0 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T12:56:20.259 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1020fc1f-1f57-42ef-b4ec-a91c53e78eb3 2024-04-25T12:56:20.260 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049779918903, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049779919343, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714049779, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1020fc1f-1f57-42ef-b4ec-a91c53e78eb3", "db_session_id": "MQ7PATNP06XO4TVHF5QL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714049779919422, "job": 1, "event": "recovery_finished"} 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557273be0000 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: DB pointer 0x557273bca000 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** DB Stats ** 2024-04-25T12:56:20.261 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:56:20.262 INFO:journalctl@ceph.mon.c.smithi178.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T12:56:20.263 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache BinnedLRUCache@0x557273acb1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: starting mon.c rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(???) e0 preinit fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T12:56:20.264 INFO:journalctl@ceph.mon.c.smithi178.stdout: e1 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout: legacy client fscid: -1 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout: No filesystems configured 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:56:20.265 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Deploying cephadm binary to smithi178 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:20.266 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Added host smithi178 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.267 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1124280651' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1124280651' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:20.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c;count:3 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:20.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: Deploying daemon mon.c on smithi178 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 ceph-mon[33233]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 bash[33187]: fd042fa8a8aaa8fe48546f7af2ab2f0d5363d8c8de3f287b47fc3897f8b7470e 2024-04-25T12:56:20.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:19 smithi178 systemd[1]: Started Ceph mon.c for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:56:24.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.956+0000 7f7c7affd700 1 -- 172.21.15.178:0/3058200832 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7c80030090 con 0x7f7c84104390 2024-04-25T12:56:24.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:56:24.955 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","modified":"2024-04-25T12:56:19.941865Z","created":"2024-04-25T12:54:36.611286Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7c70039650 msgr2=0x7f7c7003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7c70039650 0x7f7c7003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7c74006fd0 tx=0x7f7c74006e40 comp rx=0 tx=0).stop 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 msgr2=0x7f7c84197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84197fb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005780 tx=0x7f7c800057b0 comp rx=0 tx=0).stop 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 shutdown_connections 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7c70039650 0x7f7c7003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:24.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 --2- 172.21.15.178:0/3058200832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c84104390 0x7f7c84197fb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:24.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 >> 172.21.15.178:0/3058200832 conn(0x7f7c840fc5b0 msgr2=0x7f7c840fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:24.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 shutdown_connections 2024-04-25T12:56:24.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:24.959+0000 7f7c8bd57700 1 -- 172.21.15.178:0/3058200832 wait complete. 2024-04-25T12:56:24.958 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-04-25T12:56:25.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: mon.a calling monitor election 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/3058200832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: mon.c calling monitor election 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: monmap epoch 2 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: last_changed 2024-04-25T12:56:19.941865+0000 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: min_mon_release 19 (squid) 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: election_strategy: 1 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: fsmap 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: mgrmap e13: x(active, since 49s) 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: overall HEALTH_OK 2024-04-25T12:56:25.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:25.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:25.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:25.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:25.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: mon.a calling monitor election 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/3058200832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: mon.c calling monitor election 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: monmap epoch 2 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: last_changed 2024-04-25T12:56:19.941865+0000 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: min_mon_release 19 (squid) 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: election_strategy: 1 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: fsmap 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:25.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: mgrmap e13: x(active, since 49s) 2024-04-25T12:56:25.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: overall HEALTH_OK 2024-04-25T12:56:25.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:25.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:25.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:25.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:25.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:26.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:26 smithi178 ceph-mon[33233]: Deploying daemon mon.b on smithi104 2024-04-25T12:56:26.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:26.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:26 smithi038 ceph-mon[29400]: Deploying daemon mon.b on smithi104 2024-04-25T12:56:26.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:26.552 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T12:56:26.552 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T12:56:26.786 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:56:27.213 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:27 smithi178 ceph-mon[33233]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:27.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:27 smithi038 ceph-mon[29400]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 -- 172.21.15.178:0/1129557243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a481074d0 msgr2=0x7f0a481078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/1129557243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a481074d0 0x7f0a481078b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009910 tx=0x7f0a38009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 -- 172.21.15.178:0/1129557243 shutdown_connections 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/1129557243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a481074d0 0x7f0a481078b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 -- 172.21.15.178:0/1129557243 >> 172.21.15.178:0/1129557243 conn(0x7f0a480fcf70 msgr2=0x7f0a480ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 -- 172.21.15.178:0/1129557243 shutdown_connections 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.065+0000 7f0a4fed3700 1 -- 172.21.15.178:0/1129557243 wait complete. 2024-04-25T12:56:28.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4fed3700 1 Processor -- start 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4fed3700 1 -- start start 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4fed3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4fed3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a48198700 0x7f0a4819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4fed3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4810ba70 con 0x7f0a48198700 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4fed3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4810b8f0 con 0x7f0a481074d0 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57642/0 (socket says 172.21.15.178:57642) 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 -- 172.21.15.178:0/2101731991 learned_addr learned my addr 172.21.15.178:0/2101731991 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4d46e700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a48198700 0x7f0a4819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 -- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 msgr2=0x7f0a481981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 -- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 msgr2=0x7f0a481981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T12:56:28.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4dc6f700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4d46e700 1 -- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 msgr2=0x7f0a481981c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4d46e700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.066+0000 7f0a4d46e700 1 -- 172.21.15.178:0/2101731991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a48198d40 con 0x7f0a48198700 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.067+0000 7f0a4d46e700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a48198700 0x7f0a4819ca90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b4d0 tx=0x7f0a4400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.067+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0a44011630 con 0x7f0a48198700 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.067+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4819cfd0 con 0x7f0a48198700 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.067+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0a44011ca0 con 0x7f0a48198700 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.067+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0a44010e50 con 0x7f0a48198700 2024-04-25T12:56:28.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.067+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4819d530 con 0x7f0a48198700 2024-04-25T12:56:28.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.068+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a48102b80 con 0x7f0a48198700 2024-04-25T12:56:28.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.068+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0a44018020 con 0x7f0a48198700 2024-04-25T12:56:28.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.068+0000 7f0a3effd700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0a34039590 0x7f0a3403ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:28.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.068+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0a4404c190 con 0x7f0a48198700 2024-04-25T12:56:28.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.068+0000 7f0a4dc6f700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0a34039590 0x7f0a3403ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:28.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.069+0000 7f0a4dc6f700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0a34039590 0x7f0a3403ba50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800b5c0 tx=0x7f0a3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:28.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.074+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a4401f020 con 0x7f0a48198700 2024-04-25T12:56:28.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.350+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0a4819d840 con 0x7f0a48198700 2024-04-25T12:56:28.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.350+0000 7f0a3effd700 1 -- 172.21.15.178:0/2101731991 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0a44017520 con 0x7f0a48198700 2024-04-25T12:56:28.348 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:56:28.348 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","modified":"2024-04-25T12:56:19.941865Z","created":"2024-04-25T12:54:36.611286Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T12:56:28.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.352+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0a34039590 msgr2=0x7f0a3403ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.352+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0a34039590 0x7f0a3403ba50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800b5c0 tx=0x7f0a3801a040 comp rx=0 tx=0).stop 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.352+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a48198700 msgr2=0x7f0a4819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.352+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a48198700 0x7f0a4819ca90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b4d0 tx=0x7f0a4400b890 comp rx=0 tx=0).stop 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 shutdown_connections 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0a34039590 0x7f0a3403ba50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a481074d0 0x7f0a481981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 --2- 172.21.15.178:0/2101731991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a48198700 0x7f0a4819ca90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 >> 172.21.15.178:0/2101731991 conn(0x7f0a480fcf70 msgr2=0x7f0a481091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 shutdown_connections 2024-04-25T12:56:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:28.353+0000 7f0a4fed3700 1 -- 172.21.15.178:0/2101731991 wait complete. 2024-04-25T12:56:28.352 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-04-25T12:56:29.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:28 smithi104 systemd[1]: Started Ceph mon.b for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:56:29.945 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T12:56:29.946 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T12:56:30.190 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:56:34.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: mon.a calling monitor election 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: mon.c calling monitor election 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: mon.b calling monitor election 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: monmap epoch 3 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: last_changed 2024-04-25T12:56:28.899540+0000 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: min_mon_release 19 (squid) 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: election_strategy: 1 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: fsmap 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: mgrmap e13: x(active, since 58s) 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: overall HEALTH_OK 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: mon.a calling monitor election 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: mon.c calling monitor election 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: mon.b calling monitor election 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: monmap epoch 3 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: last_changed 2024-04-25T12:56:28.899540+0000 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: min_mon_release 19 (squid) 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: election_strategy: 1 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: fsmap 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: mgrmap e13: x(active, since 58s) 2024-04-25T12:56:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: overall HEALTH_OK 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: mon.a calling monitor election 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: mon.c calling monitor election 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:56:34.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: mon.b calling monitor election 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: monmap epoch 3 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: last_changed 2024-04-25T12:56:28.899540+0000 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: created 2024-04-25T12:54:36.611286+0000 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: min_mon_release 19 (squid) 2024-04-25T12:56:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: election_strategy: 1 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: fsmap 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: mgrmap e13: x(active, since 58s) 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: overall HEALTH_OK 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.534+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/2120665380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cbc0035c0 msgr2=0x7f5cbc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.534+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/2120665380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cbc0035c0 0x7f5cbc005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4102490 tx=0x7f5cc400feb0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.534+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/2120665380 shutdown_connections 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.534+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/2120665380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cbc0035c0 0x7f5cbc005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.535+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/2120665380 >> 172.21.15.178:0/2120665380 conn(0x7f5cd40fe050 msgr2=0x7f5cd4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.535+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/2120665380 shutdown_connections 2024-04-25T12:56:34.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.535+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/2120665380 wait complete. 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.535+0000 7f5cd8c5c700 1 Processor -- start 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 -- start start 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 0x7f5cd41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cd4198700 0x7f5cd419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410a610 con 0x7f5cd4198700 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410a490 con 0x7f5cbc0035c0 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd8c5c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410a790 con 0x7f5cd419cfd0 2024-04-25T12:56:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57646/0 (socket says 172.21.15.178:57646) 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 -- 172.21.15.178:0/3030301358 learned_addr learned my addr 172.21.15.178:0/3030301358 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd259c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 0x7f5cd41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 msgr2=0x7f5cd419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 msgr2=0x7f5cd419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd1d9b700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cd4198700 0x7f5cd419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd2d9d700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd259c700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 msgr2=0x7f5cd41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd259c700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 msgr2=0x7f5cd41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd259c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 0x7f5cd41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.536+0000 7f5cd259c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 0x7f5cd41981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd1d9b700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 msgr2=0x7f5cd419d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:56:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd1d9b700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd1d9b700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 msgr2=0x7f5cd41981c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd1d9b700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 0x7f5cd41981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd1d9b700 1 -- 172.21.15.178:0/3030301358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd41a4d40 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd1d9b700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cd4198700 0x7f5cd419ca90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800b550 tx=0x7f5cc800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004230 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5cc80043d0 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41a4f70 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8011bf0 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.537+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a5520 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.538+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5cc801a400 con 0x7f5cd4198700 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.538+0000 7f5cc37fe700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f5cb4039ae0 0x7f5cb403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.538+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5cc804c470 con 0x7f5cd4198700 2024-04-25T12:56:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.538+0000 7f5cd259c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f5cb4039ae0 0x7f5cb403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.538+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd40640a0 con 0x7f5cd4198700 2024-04-25T12:56:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.539+0000 7f5cd259c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f5cb4039ae0 0x7f5cb403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005b40 tx=0x7f5cc4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:34.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.544+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cc8003c90 con 0x7f5cd4198700 2024-04-25T12:56:34.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.852+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5cd41a57e0 con 0x7f5cd4198700 2024-04-25T12:56:34.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.852+0000 7f5cc37fe700 1 -- 172.21.15.178:0/3030301358 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f5cc801a790 con 0x7f5cd4198700 2024-04-25T12:56:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T12:56:34.851 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","modified":"2024-04-25T12:56:28.899540Z","created":"2024-04-25T12:54:36.611286Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T12:56:34.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f5cb4039ae0 msgr2=0x7f5cb403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f5cb4039ae0 0x7f5cb403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005b40 tx=0x7f5cc4005a90 comp rx=0 tx=0).stop 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cd4198700 msgr2=0x7f5cd419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cd4198700 0x7f5cd419ca90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800b550 tx=0x7f5cc800b910 comp rx=0 tx=0).stop 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 shutdown_connections 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f5cb4039ae0 0x7f5cb403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cbc0035c0 0x7f5cd41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd419cfd0 0x7f5cd419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 --2- 172.21.15.178:0/3030301358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cd4198700 0x7f5cd419ca90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 >> 172.21.15.178:0/3030301358 conn(0x7f5cd40fe050 msgr2=0x7f5cd40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 shutdown_connections 2024-04-25T12:56:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:34.855+0000 7f5cd8c5c700 1 -- 172.21.15.178:0/3030301358 wait complete. 2024-04-25T12:56:34.854 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-04-25T12:56:35.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:34 smithi178 ceph-mon[33233]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:35.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:34 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:35.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:34 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/3030301358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:56:35.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:34 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:35.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:34 smithi038 ceph-mon[29400]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:35.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:34 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:35.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:34 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/3030301358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:56:35.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:34 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:35.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:34 smithi104 ceph-mon[32726]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:35.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:34 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:35.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:34 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/3030301358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T12:56:35.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:34 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:56:35.582 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T12:56:35.583 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T12:56:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:36.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:36.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T12:56:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T12:56:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:36 smithi038 ceph-mon[29400]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T12:56:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:36 smithi104 ceph-mon[32726]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T12:56:36.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:36.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T12:56:36.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:36 smithi178 ceph-mon[33233]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T12:56:37.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T12:56:37.548 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:37.548 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:37.548 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:37.548 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:37 smithi038 ceph-mon[29400]: Reconfiguring daemon mon.a on smithi038 2024-04-25T12:56:37.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T12:56:37.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:37.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:37.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:37.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:37 smithi178 ceph-mon[33233]: Reconfiguring daemon mon.a on smithi038 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:37.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:37 smithi104 ceph-mon[32726]: Reconfiguring daemon mon.a on smithi038 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 -- 172.21.15.38:0/722270109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1015f0 msgr2=0x7f417c1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 --2- 172.21.15.38:0/722270109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1015f0 0x7f417c1019d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4178009910 tx=0x7f4178009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 -- 172.21.15.38:0/722270109 shutdown_connections 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 --2- 172.21.15.38:0/722270109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1015f0 0x7f417c1019d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 -- 172.21.15.38:0/722270109 >> 172.21.15.38:0/722270109 conn(0x7f417c076ea0 msgr2=0x7f417c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 -- 172.21.15.38:0/722270109 shutdown_connections 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.451+0000 7f41840d9700 1 -- 172.21.15.38:0/722270109 wait complete. 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 Processor -- start 2024-04-25T12:56:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 -- start start 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f417c1015f0 0x7f417c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f417c103580 0x7f417c103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 0x7f417c1a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10b920 con 0x7f417c1a43b0 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10b7a0 con 0x7f417c103580 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f41840d9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10baa0 con 0x7f417c1015f0 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f4182676700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 0x7f417c1a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f4182676700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 0x7f417c1a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45432/0 (socket says 172.21.15.38:45432) 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f4182676700 1 -- 172.21.15.38:0/1813146772 learned_addr learned my addr 172.21.15.38:0/1813146772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.452+0000 7f4181674700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f417c103580 0x7f417c103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4182676700 1 -- 172.21.15.38:0/1813146772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f417c1015f0 msgr2=0x7f417c103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4182676700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f417c1015f0 0x7f417c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4182676700 1 -- 172.21.15.38:0/1813146772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f417c103580 msgr2=0x7f417c103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4182676700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f417c103580 0x7f417c103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4182676700 1 -- 172.21.15.38:0/1813146772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c1a6ed0 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4181674700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f417c103580 0x7f417c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4182676700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 0x7f417c1a67a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f417400ba50 tx=0x7f417400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174019070 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4174014440 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c1a71c0 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174007570 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.453+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1a76b0 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.454+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f417401c070 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.454+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c0640a0 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.454+0000 7f4172ffd700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4168039ae0 0x7f416803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.454+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f417404bf90 con 0x7f417c1a43b0 2024-04-25T12:56:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.454+0000 7f4181e75700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4168039ae0 0x7f416803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.455+0000 7f4181e75700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4168039ae0 0x7f416803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f417800b5c0 tx=0x7f4178005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.459+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41740149a0 con 0x7f417c1a43b0 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.673+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f417c1a7a10 con 0x7f417c1a43b0 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T12:56:38.068 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.673+0000 7f4172ffd700 1 -- 172.21.15.38:0/1813146772 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f4174007710 con 0x7f417c1a43b0 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4168039ae0 msgr2=0x7f416803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4168039ae0 0x7f416803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f417800b5c0 tx=0x7f4178005fd0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 msgr2=0x7f417c1a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 0x7f417c1a67a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f417400ba50 tx=0x7f417400be10 comp rx=0 tx=0).stop 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 shutdown_connections 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4168039ae0 0x7f416803bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f417c1a43b0 0x7f417c1a67a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f417c1015f0 0x7f417c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 --2- 172.21.15.38:0/1813146772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f417c103580 0x7f417c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 >> 172.21.15.38:0/1813146772 conn(0x7f417c076ea0 msgr2=0x7f417c1092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 shutdown_connections 2024-04-25T12:56:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:37.675+0000 7f41840d9700 1 -- 172.21.15.38:0/1813146772 wait complete. 2024-04-25T12:56:38.578 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:38 smithi038 ceph-mon[29400]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:38.578 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:38 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1813146772' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:38.612 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T12:56:38.613 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:56:38.613 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T12:56:38.643 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:56:38.643 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:38.709 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:56:38.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T12:56:38.742 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:56:38.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:38.805 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:38 smithi104 ceph-mon[32726]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:38.805 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:38 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1813146772' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:38.809 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:56:38.809 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T12:56:38.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:38 smithi178 ceph-mon[33233]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:38.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:38 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1813146772' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:38.840 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:56:38.841 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:56:38.908 INFO:tasks.cephadm:Adding mgr.x on smithi038 2024-04-25T12:56:38.908 INFO:tasks.cephadm:Adding mgr.y on smithi104 2024-04-25T12:56:38.909 INFO:tasks.cephadm:Adding mgr.z on smithi178 2024-04-25T12:56:38.909 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi038=x;smithi104=y;smithi178=z' 2024-04-25T12:56:39.184 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:56:40.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:40.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: Reconfiguring mon.b (monmap changed)... 2024-04-25T12:56:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:40 smithi038 ceph-mon[29400]: Reconfiguring daemon mon.b on smithi104 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: Reconfiguring mon.b (monmap changed)... 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:40.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:40 smithi104 ceph-mon[32726]: Reconfiguring daemon mon.b on smithi104 2024-04-25T12:56:40.326 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:40.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:40.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: Reconfiguring mon.b (monmap changed)... 2024-04-25T12:56:40.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:40.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:40.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:40.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:40 smithi178 ceph-mon[33233]: Reconfiguring daemon mon.b on smithi104 2024-04-25T12:56:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 -- 172.21.15.178:0/3387046124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 msgr2=0x7f08b8101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/3387046124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 0x7f08b8101910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009910 tx=0x7f08a8009c20 comp rx=0 tx=0).stop 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 -- 172.21.15.178:0/3387046124 shutdown_connections 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/3387046124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 0x7f08b8101910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 -- 172.21.15.178:0/3387046124 >> 172.21.15.178:0/3387046124 conn(0x7f08b80fca70 msgr2=0x7f08b80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 -- 172.21.15.178:0/3387046124 shutdown_connections 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.714+0000 7f08bc8c7700 1 -- 172.21.15.178:0/3387046124 wait complete. 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 Processor -- start 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 -- start start 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 0x7f08b806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b819ab50 0x7f08b819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 0x7f08b819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8107520 con 0x7f08b81014f0 2024-04-25T12:56:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81073a0 con 0x7f08b819ab50 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.715+0000 7f08bc8c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81076a0 con 0x7f08b819d4d0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 0x7f08b819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 0x7f08b819f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52614/0 (socket says 172.21.15.178:52614) 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 -- 172.21.15.178:0/506771691 learned_addr learned my addr 172.21.15.178:0/506771691 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b659c700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 0x7f08b806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b5d9b700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b819ab50 0x7f08b819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 -- 172.21.15.178:0/506771691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b819ab50 msgr2=0x7f08b819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b819ab50 0x7f08b819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 -- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 msgr2=0x7f08b806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 0x7f08b806b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 -- 172.21.15.178:0/506771691 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b81a0230 con 0x7f08b819d4d0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08b6d9d700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 0x7f08b819f960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08b000ba50 tx=0x7f08b000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0019070 con 0x7f08b819d4d0 2024-04-25T12:56:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b81a0520 con 0x7f08b819d4d0 2024-04-25T12:56:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f08b000ce00 con 0x7f08b819d4d0 2024-04-25T12:56:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0007350 con 0x7f08b819d4d0 2024-04-25T12:56:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.716+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b81a0ae0 con 0x7f08b819d4d0 2024-04-25T12:56:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.717+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f08b001c070 con 0x7f08b819d4d0 2024-04-25T12:56:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.717+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0898005360 con 0x7f08b819d4d0 2024-04-25T12:56:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.718+0000 7f08a77fe700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f08a0039ae0 0x7f08a003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.718+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f08b004bcb0 con 0x7f08b819d4d0 2024-04-25T12:56:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.718+0000 7f08b659c700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f08a0039ae0 0x7f08a003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.718+0000 7f08b659c700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f08a0039ae0 0x7f08a003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005230 tx=0x7f08a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.723+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08b0007db0 con 0x7f08b819d4d0 2024-04-25T12:56:40.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.941+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi104=y;smithi178=z", "target": ["mon-mgr", ""]}) -- 0x7f0898000bf0 con 0x7f08a0039ae0 2024-04-25T12:56:40.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.951+0000 7f08a77fe700 1 -- 172.21.15.178:0/506771691 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0898000bf0 con 0x7f08a0039ae0 2024-04-25T12:56:40.950 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f08a0039ae0 msgr2=0x7f08a003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f08a0039ae0 0x7f08a003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005230 tx=0x7f08a801a040 comp rx=0 tx=0).stop 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 msgr2=0x7f08b819f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 0x7f08b819f960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08b000ba50 tx=0x7f08b000bd60 comp rx=0 tx=0).stop 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 shutdown_connections 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f08a0039ae0 0x7f08a003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08b819d4d0 0x7f08b819f960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81014f0 0x7f08b806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 --2- 172.21.15.178:0/506771691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b819ab50 0x7f08b819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 >> 172.21.15.178:0/506771691 conn(0x7f08b80fca70 msgr2=0x7f08b80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 shutdown_connections 2024-04-25T12:56:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:56:40.954+0000 7f08bc8c7700 1 -- 172.21.15.178:0/506771691 wait complete. 2024-04-25T12:56:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.529 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T12:56:41.532 DEBUG:teuthology.orchestra.run.smithi178:mgr.z> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T12:56:41.534 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T12:56:41.534 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T12:56:41.534 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:56:41.555 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:56:41.555 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-04-25T12:56:41.556 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:41.556 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.556 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.556 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:41.556 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:41.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:41.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T12:56:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T12:56:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:41.588 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:56:41.588 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:41.588 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T12:56:41.589 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:41.589 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:41.589 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 12:55:45.652470597 +0000 2024-04-25T12:56:41.589 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 12:51:15.268117337 +0000 2024-04-25T12:56:41.589 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 12:51:15.268117337 +0000 2024-04-25T12:56:41.589 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 12:51:15.268117337 +0000 2024-04-25T12:56:41.589 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:56:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T12:56:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T12:56:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000320758 s, 1.6 MB/s 2024-04-25T12:56:41.658 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:56:41.717 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-04-25T12:56:41.775 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:56:41.776 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:41.776 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T12:56:41.776 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:41.776 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:41.776 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 12:55:45.679470073 +0000 2024-04-25T12:56:41.777 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 12:51:15.616110387 +0000 2024-04-25T12:56:41.777 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 12:51:15.616110387 +0000 2024-04-25T12:56:41.777 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 12:51:15.616110387 +0000 2024-04-25T12:56:41.777 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:56:41.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T12:56:41.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T12:56:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000281602 s, 1.8 MB/s 2024-04-25T12:56:41.845 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:56:41.904 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-04-25T12:56:41.961 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 12:55:45.705469568 +0000 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 12:51:15.982103078 +0000 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 12:51:15.982103078 +0000 2024-04-25T12:56:41.962 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 12:51:15.982103078 +0000 2024-04-25T12:56:41.963 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:56:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T12:56:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T12:56:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000295814 s, 1.7 MB/s 2024-04-25T12:56:42.028 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:56:42.086 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 12:55:45.731469063 +0000 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 12:51:16.330096129 +0000 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 12:51:16.330096129 +0000 2024-04-25T12:56:42.143 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 12:51:16.330096129 +0000 2024-04-25T12:56:42.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:56:42.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T12:56:42.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T12:56:42.208 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000286326 s, 1.8 MB/s 2024-04-25T12:56:42.210 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:56:42.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:42 smithi178 ceph-mon[33233]: Reconfiguring mon.c (monmap changed)... 2024-04-25T12:56:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:42 smithi178 ceph-mon[33233]: Reconfiguring daemon mon.c on smithi178 2024-04-25T12:56:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:42 smithi178 ceph-mon[33233]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi104=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:42 smithi178 ceph-mon[33233]: Saving service mgr spec with placement smithi038=x;smithi104=y;smithi178=z;count:3 2024-04-25T12:56:42.267 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-25T12:56:42.268 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:56:42.287 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:56:42.287 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-04-25T12:56:42.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:42 smithi038 ceph-mon[29400]: Reconfiguring mon.c (monmap changed)... 2024-04-25T12:56:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:42 smithi038 ceph-mon[29400]: Reconfiguring daemon mon.c on smithi178 2024-04-25T12:56:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:42 smithi038 ceph-mon[29400]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi104=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:42 smithi038 ceph-mon[29400]: Saving service mgr spec with placement smithi038=x;smithi104=y;smithi178=z;count:3 2024-04-25T12:56:42.344 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:56:42.344 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:42.344 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T12:56:42.344 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:42.345 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:42.345 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-25 12:56:28.230565867 +0000 2024-04-25T12:56:42.345 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-25 12:51:27.714514900 +0000 2024-04-25T12:56:42.345 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-25 12:51:27.714514900 +0000 2024-04-25T12:56:42.345 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-25 12:51:27.714514900 +0000 2024-04-25T12:56:42.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:56:42.409 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:42 smithi104 ceph-mon[32726]: Reconfiguring mon.c (monmap changed)... 2024-04-25T12:56:42.409 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:42 smithi104 ceph-mon[32726]: Reconfiguring daemon mon.c on smithi178 2024-04-25T12:56:42.409 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:42 smithi104 ceph-mon[32726]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi104=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:42.409 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:42 smithi104 ceph-mon[32726]: Saving service mgr spec with placement smithi038=x;smithi104=y;smithi178=z;count:3 2024-04-25T12:56:42.411 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-25T12:56:42.411 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-25T12:56:42.411 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000295664 s, 1.7 MB/s 2024-04-25T12:56:42.412 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:56:42.470 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-04-25T12:56:42.527 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:56:42.527 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:42.527 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T12:56:42.527 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:42.528 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:42.528 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-25 12:56:28.231565856 +0000 2024-04-25T12:56:42.528 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-25 12:51:28.061510852 +0000 2024-04-25T12:56:42.528 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-25 12:51:28.061510852 +0000 2024-04-25T12:56:42.528 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-25 12:51:28.061510852 +0000 2024-04-25T12:56:42.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:56:42.592 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-25T12:56:42.593 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-25T12:56:42.593 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000204342 s, 2.5 MB/s 2024-04-25T12:56:42.594 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:56:42.651 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-04-25T12:56:42.707 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:56:42.707 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:42.707 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T12:56:42.707 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:42.707 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:42.707 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-25 12:56:28.231565856 +0000 2024-04-25T12:56:42.708 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-25 12:51:28.414506734 +0000 2024-04-25T12:56:42.708 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-25 12:51:28.414506734 +0000 2024-04-25T12:56:42.708 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-25 12:51:28.414506734 +0000 2024-04-25T12:56:42.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:56:42.770 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-25T12:56:42.771 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-25T12:56:42.771 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000221894 s, 2.3 MB/s 2024-04-25T12:56:42.772 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:56:42.828 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-04-25T12:56:42.884 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-25 12:56:28.232565845 +0000 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-25 12:51:28.738502955 +0000 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-25 12:51:28.738502955 +0000 2024-04-25T12:56:42.885 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-25 12:51:28.738502955 +0000 2024-04-25T12:56:42.885 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:56:42.948 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-25T12:56:42.948 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-25T12:56:42.948 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000230158 s, 2.2 MB/s 2024-04-25T12:56:42.949 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:56:43.005 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T12:56:43.005 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:56:43.023 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:56:43.023 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-04-25T12:56:43.079 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:56:43.079 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:43.079 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T12:56:43.079 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:43.079 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:43.079 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-25 12:56:18.884542143 +0000 2024-04-25T12:56:43.080 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-25 12:51:29.761587472 +0000 2024-04-25T12:56:43.080 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-25 12:51:29.761587472 +0000 2024-04-25T12:56:43.080 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-25 12:51:29.761587472 +0000 2024-04-25T12:56:43.080 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:56:43.146 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-25T12:56:43.146 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-25T12:56:43.146 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000125123 s, 4.1 MB/s 2024-04-25T12:56:43.147 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:56:43.205 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-04-25T12:56:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:56:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:43.260 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T12:56:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-25 12:56:18.885542125 +0000 2024-04-25T12:56:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-25 12:51:30.090580995 +0000 2024-04-25T12:56:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-25 12:51:30.090580995 +0000 2024-04-25T12:56:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-25 12:51:30.090580995 +0000 2024-04-25T12:56:43.261 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:56:43.323 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-25T12:56:43.323 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-25T12:56:43.323 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.694e-05 s, 5.3 MB/s 2024-04-25T12:56:43.324 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:56:43.380 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-04-25T12:56:43.435 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:56:43.435 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-25 12:56:18.885542125 +0000 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-25 12:51:30.424574420 +0000 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-25 12:51:30.424574420 +0000 2024-04-25T12:56:43.436 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-25 12:51:30.424574420 +0000 2024-04-25T12:56:43.436 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:56:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:43 smithi178 ceph-mon[33233]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:43.498 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-25T12:56:43.498 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-25T12:56:43.498 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000116019 s, 4.4 MB/s 2024-04-25T12:56:43.499 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:56:43.555 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-04-25T12:56:43.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:43 smithi038 ceph-mon[29400]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:43.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:43.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:43.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:43 smithi104 ceph-mon[32726]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:43.610 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:56:43.610 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-25 12:56:18.885542125 +0000 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-25 12:51:30.746568081 +0000 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-25 12:51:30.746568081 +0000 2024-04-25T12:56:43.611 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-25 12:51:30.746568081 +0000 2024-04-25T12:56:43.611 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:56:43.673 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-25T12:56:43.673 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-25T12:56:43.673 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.6896e-05 s, 5.3 MB/s 2024-04-25T12:56:43.675 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:56:43.731 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/vg_nvme/lv_4... 2024-04-25T12:56:43.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T12:56:43.978 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:44.195 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:44.196 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:44 smithi178 ceph-mon[33233]: Deploying daemon mgr.z on smithi178 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:44.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:44 smithi038 ceph-mon[29400]: Deploying daemon mgr.z on smithi178 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:56:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:44 smithi104 ceph-mon[32726]: Deploying daemon mgr.z on smithi178 2024-04-25T12:56:45.424 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 systemd[1]: Starting Ceph mgr.z for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T12:56:45.719 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 podman[35087]: 2024-04-25 12:56:45.32964197 +0000 UTC m=+0.018071875 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:56:45.720 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 podman[35087]: 2024-04-25 12:56:45.436039758 +0000 UTC m=+0.124469700 container create 07d8327823261f08e816cfbde1cb713a4481817f3cb5feafc9b82e0ffab8e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T12:56:45.720 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 podman[35087]: 2024-04-25 12:56:45.63871806 +0000 UTC m=+0.327147956 container init 07d8327823261f08e816cfbde1cb713a4481817f3cb5feafc9b82e0ffab8e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:56:45.720 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 podman[35087]: 2024-04-25 12:56:45.641199633 +0000 UTC m=+0.329629540 container start 07d8327823261f08e816cfbde1cb713a4481817f3cb5feafc9b82e0ffab8e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T12:56:45.720 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 ceph-mgr[35107]: -- 172.21.15.178:0/3520133741 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624747485a0 con 0x56247471c800 2024-04-25T12:56:45.720 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 bash[35087]: 07d8327823261f08e816cfbde1cb713a4481817f3cb5feafc9b82e0ffab8e768 2024-04-25T12:56:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T12:56:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T12:56:45.889 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-25T12:56:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-25T12:56:45.889 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0100859 s, 1.0 GB/s 2024-04-25T12:56:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-25T12:56:45.976 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 systemd[1]: Started Ceph mgr.z for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:56:45.976 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:45.793+0000 7fef2ee5d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:56:45.976 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:45.977+0000 7fef2ee5d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:56:46.238 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:46.079+0000 7fef2ee5d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:56:46.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-04-25T12:56:46.738 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:46.386+0000 7fef2ee5d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:56:46.738 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:46.470+0000 7fef2ee5d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:56:46.783 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:56:47.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:47.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:56:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:46 smithi038 ceph-mon[29400]: Deploying daemon mgr.y on smithi104 2024-04-25T12:56:47.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:47.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:47.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:56:47.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:47.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:47.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:46 smithi104 ceph-mon[32726]: Deploying daemon mgr.y on smithi104 2024-04-25T12:56:47.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:47.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T12:56:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:46 smithi178 ceph-mon[33233]: Deploying daemon mgr.y on smithi104 2024-04-25T12:56:47.988 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:47.667+0000 7fef2ee5d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:56:47.988 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:47.836+0000 7fef2ee5d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:56:47.990 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:47 smithi104 systemd[1]: Starting Ceph mgr.y for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T12:56:48.245 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:47 smithi104 podman[34097]: 2024-04-25 12:56:47.895766804 +0000 UTC m=+0.017882246 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:56:48.245 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 podman[34097]: 2024-04-25 12:56:48.016904349 +0000 UTC m=+0.139019835 container create f44d51b5520cae9dc9ffdc83b29a64f2e2566c0ed525b291c2b59214a57aaa65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T12:56:48.245 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 podman[34097]: 2024-04-25 12:56:48.219632454 +0000 UTC m=+0.341747903 container init f44d51b5520cae9dc9ffdc83b29a64f2e2566c0ed525b291c2b59214a57aaa65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T12:56:48.245 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 podman[34097]: 2024-04-25 12:56:48.222383848 +0000 UTC m=+0.344499301 container start f44d51b5520cae9dc9ffdc83b29a64f2e2566c0ed525b291c2b59214a57aaa65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:56:48.245 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mgr[34118]: -- 172.21.15.104:0/2405152910 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636616b85a0 con 0x56366168cc00 2024-04-25T12:56:48.315 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:48.015+0000 7fef2ee5d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 -- 172.21.15.38:0/2688418352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28381019a0 msgr2=0x7f2838101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 --2- 172.21.15.38:0/2688418352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28381019a0 0x7f2838101dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28280099c0 tx=0x7f2828009cd0 comp rx=0 tx=0).stop 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 -- 172.21.15.38:0/2688418352 shutdown_connections 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 --2- 172.21.15.38:0/2688418352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838103540 0x7f2838109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 --2- 172.21.15.38:0/2688418352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28381019a0 0x7f2838101dc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 --2- 172.21.15.38:0/2688418352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2838102ba0 0x7f2838103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 -- 172.21.15.38:0/2688418352 >> 172.21.15.38:0/2688418352 conn(0x7f28380fcf40 msgr2=0x7f28380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 -- 172.21.15.38:0/2688418352 shutdown_connections 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.505+0000 7f284000d700 1 -- 172.21.15.38:0/2688418352 wait complete. 2024-04-25T12:56:48.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 Processor -- start 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 -- start start 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28381019a0 0x7f2838198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838102ba0 0x7f2838199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 0x7f28381a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f283810d550 con 0x7f2838103540 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f283810d3d0 con 0x7f2838102ba0 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f284000d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f283810d6d0 con 0x7f28381019a0 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f283e5aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 0x7f28381a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f283d5a8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838102ba0 0x7f2838199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f283dda9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28381019a0 0x7f2838198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f283dda9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28381019a0 0x7f2838198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.38:39840/0 (socket says 172.21.15.38:39840) 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f283dda9700 1 -- 172.21.15.38:0/1699055315 learned_addr learned my addr 172.21.15.38:0/1699055315 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:56:48.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.506+0000 7f283e5aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 0x7f28381a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40582/0 (socket says 172.21.15.38:40582) 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283e5aa700 1 -- 172.21.15.38:0/1699055315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28381019a0 msgr2=0x7f2838198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283e5aa700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28381019a0 0x7f2838198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283e5aa700 1 -- 172.21.15.38:0/1699055315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838102ba0 msgr2=0x7f2838199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283e5aa700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838102ba0 0x7f2838199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283e5aa700 1 -- 172.21.15.38:0/1699055315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28381a0940 con 0x7f2838103540 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283d5a8700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838102ba0 0x7f2838199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f283e5aa700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 0x7f28381a0270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f283000efc0 tx=0x7f283000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300092e0 con 0x7f2838103540 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2830004bb0 con 0x7f2838103540 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28381a0bd0 con 0x7f2838103540 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830004280 con 0x7f2838103540 2024-04-25T12:56:48.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.507+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28381a1110 con 0x7f2838103540 2024-04-25T12:56:48.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.508+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f283001b030 con 0x7f2838103540 2024-04-25T12:56:48.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.508+0000 7f282effd700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2824039890 0x7f282403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:56:48.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.508+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f283004c4a0 con 0x7f2838103540 2024-04-25T12:56:48.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.508+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2838069730 con 0x7f2838103540 2024-04-25T12:56:48.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.508+0000 7f283dda9700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2824039890 0x7f282403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:56:48.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.509+0000 7f283dda9700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2824039890 0x7f282403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f28280038a0 tx=0x7f2828005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:56:48.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.514+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2830022020 con 0x7f2838103540 2024-04-25T12:56:48.566 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 bash[34097]: f44d51b5520cae9dc9ffdc83b29a64f2e2566c0ed525b291c2b59214a57aaa65 2024-04-25T12:56:48.566 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 systemd[1]: Started Ceph mgr.y for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T12:56:48.566 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:48.376+0000 7f25f7f5f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:56:48.620 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:48.317+0000 7fef2ee5d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:56:48.620 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:48.411+0000 7fef2ee5d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:56:48.620 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:48.622+0000 7fef2ee5d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:56:48.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:48.731+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f28380ff150 con 0x7f2824039890 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:48.571+0000 7f25f7f5f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:48.680+0000 7f25f7f5f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:48.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:48.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:48.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:48.988 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:48.847+0000 7fef2ee5d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:56:48.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:48.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:48.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:49.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:49.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:49.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:49.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:49.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:49.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:49.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:56:49.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:56:49.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:49.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:49.008+0000 7f25f7f5f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:56:49.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:49.116+0000 7f25f7f5f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:56:49.739 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:49.496+0000 7fef2ee5d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:56:49.937 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:49 smithi038 ceph-mon[29400]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:50.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:49 smithi104 ceph-mon[32726]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:50.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:49 smithi178 ceph-mon[33233]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:56:50.241 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:49.939+0000 7fef2ee5d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:56:50.241 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:50.037+0000 7fef2ee5d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:56:50.677 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:50.344+0000 7f25f7f5f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:56:50.677 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:50.511+0000 7f25f7f5f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:56:50.677 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:50.678+0000 7f25f7f5f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:56:50.738 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:50.241+0000 7fef2ee5d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:56:50.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:50.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:50.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:50.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:50.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:50.767+0000 7fef2ee5d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:50.859+0000 7fef2ee5d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:51.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:51.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:51.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:51.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:51.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.251 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:50.963+0000 7f25f7f5f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:56:51.252 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:51.054+0000 7f25f7f5f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:56:51.252 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:51.253+0000 7f25f7f5f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:56:51.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:51.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:51.441 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:51.101+0000 7fef2ee5d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:56:51.442 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:51.233+0000 7fef2ee5d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:56:51.442 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:51.327+0000 7fef2ee5d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:56:51.442 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:51.442+0000 7fef2ee5d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:56:51.573 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:51.443+0000 7f25f7f5f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:56:52.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:51 smithi038 ceph-mon[29400]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T12:56:52.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:51 smithi038 ceph-mon[29400]: Reconfiguring daemon mgr.x on smithi038 2024-04-25T12:56:52.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:51 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/118454396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc0ea8fe-71f4-4976-8844-d906980de942"}]: dispatch 2024-04-25T12:56:52.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:51 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/118454396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc0ea8fe-71f4-4976-8844-d906980de942"}]': finished 2024-04-25T12:56:52.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:51 smithi038 ceph-mon[29400]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T12:56:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:56:52.238 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:51.748+0000 7fef2ee5d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:56:52.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-mon[33233]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T12:56:52.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-mon[33233]: Reconfiguring daemon mgr.x on smithi038 2024-04-25T12:56:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/118454396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc0ea8fe-71f4-4976-8844-d906980de942"}]: dispatch 2024-04-25T12:56:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/118454396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc0ea8fe-71f4-4976-8844-d906980de942"}]': finished 2024-04-25T12:56:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-mon[33233]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T12:56:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:51 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:52.032+0000 7f25f7f5f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-mon[32726]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-mon[32726]: Reconfiguring daemon mgr.x on smithi038 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/118454396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc0ea8fe-71f4-4976-8844-d906980de942"}]: dispatch 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/118454396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc0ea8fe-71f4-4976-8844-d906980de942"}]': finished 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-mon[32726]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T12:56:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:51 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:56:52.734 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:52.446+0000 7f25f7f5f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:56:52.734 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:52.540+0000 7f25f7f5f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:56:52.735 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:52.736+0000 7f25f7f5f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:56:52.738 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:52.351+0000 7fef2ee5d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:56:52.738 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T12:56:52.567+0000 7fef2ee5d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:56:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:52.935+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f2830012050 con 0x7f2838103540 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2939250014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: Standby manager daemon z started 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:56:53.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:52 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:56:53.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:53.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2939250014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:53.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: Standby manager daemon z started 2024-04-25T12:56:53.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:56:53.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:56:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:56:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:52 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:56:53.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:53.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2939250014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:56:53.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: Standby manager daemon z started 2024-04-25T12:56:53.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:56:53.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:56:53.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:56:53.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:52 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/3158194207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:56:53.573 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:53.260+0000 7f25f7f5f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:56:53.573 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:53.351+0000 7f25f7f5f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:56:53.929 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:53.592+0000 7f25f7f5f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:56:53.930 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:53.722+0000 7f25f7f5f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:56:53.930 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:53.815+0000 7f25f7f5f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:56:53.930 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:53.931+0000 7f25f7f5f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:56:54.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:53 smithi178 ceph-mon[33233]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T12:56:54.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:56:54.256 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-mon[32726]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T12:56:54.256 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:56:54.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:53 smithi038 ceph-mon[29400]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T12:56:54.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:56:54.572 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:54.257+0000 7f25f7f5f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:56:55.222 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:54 smithi038 ceph-mon[29400]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:55.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:54 smithi178 ceph-mon[33233]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:55.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:54 smithi104 ceph-mon[32726]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:55.323 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:54.849+0000 7f25f7f5f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:56:55.323 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 12:56:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T12:56:55.061+0000 7f25f7f5f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:56:55.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:56:55.956+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2830007500 con 0x7f2838103540 2024-04-25T12:56:56.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:55 smithi038 ceph-mon[29400]: Standby manager daemon y started 2024-04-25T12:56:56.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:55 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:56:56.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:55 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:56:56.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:55 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:56:56.161 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:55 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:56:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:55 smithi178 ceph-mon[33233]: Standby manager daemon y started 2024-04-25T12:56:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:55 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:56:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:55 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:56:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:55 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:56:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:55 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:56:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:55 smithi104 ceph-mon[32726]: Standby manager daemon y started 2024-04-25T12:56:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:55 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:56:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:55 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:56:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:55 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:56:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:55 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/1955063194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:56:57.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:56 smithi038 ceph-mon[29400]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:57.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:56 smithi038 ceph-mon[29400]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T12:56:57.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:56:57.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:57.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:57.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:56 smithi178 ceph-mon[33233]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:57.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:56 smithi178 ceph-mon[33233]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T12:56:57.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:56:57.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:57.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:56 smithi104 ceph-mon[32726]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:56 smithi104 ceph-mon[32726]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T12:56:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:56:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:58.002 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:58.002 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:58.003 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:58.003 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:58.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:58.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:58.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:58.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:56:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:56:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:56:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:56:59.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:58 smithi178 ceph-mon[33233]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:59.301 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:58 smithi038 ceph-mon[29400]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:56:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:58 smithi104 ceph-mon[32726]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:00.295 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:59 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T12:57:00.296 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:56:59 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:00.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:59 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T12:57:00.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:56:59 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:00.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:59 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T12:57:00.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:56:59 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:01.247 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:00 smithi038 ceph-mon[29400]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:01.247 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:00 smithi038 ceph-mon[29400]: Deploying daemon osd.0 on smithi038 2024-04-25T12:57:01.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:00 smithi104 ceph-mon[32726]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:00 smithi104 ceph-mon[32726]: Deploying daemon osd.0 on smithi038 2024-04-25T12:57:01.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:00 smithi178 ceph-mon[33233]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:01.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:00 smithi178 ceph-mon[33233]: Deploying daemon osd.0 on smithi038 2024-04-25T12:57:03.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:02 smithi038 ceph-mon[29400]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:03.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:03 smithi104 ceph-mon[32726]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:03.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:02 smithi178 ceph-mon[33233]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:05 smithi038 ceph-mon[29400]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:05 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:05.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:05 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:05.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:05 smithi104 ceph-mon[32726]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:05.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:05 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:05.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:05 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:05.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:05 smithi178 ceph-mon[33233]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:05.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:05 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:05.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:05 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:06.315 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:06.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:06.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:07.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:07 smithi038 ceph-mon[29400]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:07.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:07 smithi104 ceph-mon[32726]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:07.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:07 smithi178 ceph-mon[33233]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:08.487 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-04-25T12:57:08.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.488+0000 7f282effd700 1 -- 172.21.15.38:0/1699055315 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f28380ff150 con 0x7f2824039890 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2824039890 msgr2=0x7f282403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2824039890 0x7f282403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f28280038a0 tx=0x7f2828005fb0 comp rx=0 tx=0).stop 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 msgr2=0x7f28381a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 0x7f28381a0270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f283000efc0 tx=0x7f283000c3a0 comp rx=0 tx=0).stop 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 shutdown_connections 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2824039890 0x7f282403bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2838103540 0x7f28381a0270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28381019a0 0x7f2838198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:08.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 --2- 172.21.15.38:0/1699055315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2838102ba0 0x7f2838199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:08.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 >> 172.21.15.38:0/1699055315 conn(0x7f28380fcf40 msgr2=0x7f28380fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:08.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 shutdown_connections 2024-04-25T12:57:08.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:57:08.491+0000 7f284000d700 1 -- 172.21.15.38:0/1699055315 wait complete. 2024-04-25T12:57:09.130 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:09 smithi038 ceph-mon[29400]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:09.130 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:09 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:09.130 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:09 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:09.130 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:09 smithi038 ceph-mon[29400]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T12:57:09.163 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T12:57:09.166 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2024-04-25T12:57:09.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T12:57:09.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:09 smithi104 ceph-mon[32726]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:09.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:09 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:09.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:09 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:09.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:09 smithi104 ceph-mon[32726]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T12:57:09.413 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:57:09.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:09 smithi178 ceph-mon[33233]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:09.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:09 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:09.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:09 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:09.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:09 smithi178 ceph-mon[33233]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:57:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:10.553 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:10.553 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:10.553 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:10.553 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T12:57:10.554 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T12:57:10.554 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:57:10.554 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:57:10.557 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.166 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0[41521]: 2024-04-25T12:57:10.896+0000 7f526db90700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T12:57:11.166 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:57:10 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0[41521]: 2024-04-25T12:57:10.900+0000 7f526597b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:57:11.342 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T12:57:11.343 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T12:57:11.343 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-25T12:57:11.343 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-25T12:57:11.343 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114409 s, 917 MB/s 2024-04-25T12:57:11.343 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-25T12:57:11.642 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:11 smithi104 ceph-mon[32726]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:57:11.642 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:11 smithi104 ceph-mon[32726]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T12:57:11.642 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:11 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.642 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:11 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.642 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:11 smithi104 ceph-mon[32726]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' 2024-04-25T12:57:11.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:11 smithi178 ceph-mon[33233]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:57:11.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:11 smithi178 ceph-mon[33233]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T12:57:11.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:11 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:11 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:11 smithi178 ceph-mon[33233]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' 2024-04-25T12:57:11.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:11 smithi038 ceph-mon[29400]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:57:11.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:11 smithi038 ceph-mon[29400]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T12:57:11.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:11 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:11 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:11.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:11 smithi038 ceph-mon[29400]: from='osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171]' entity='osd.0' 2024-04-25T12:57:11.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-04-25T12:57:12.208 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:57:12.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:57:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:57:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] boot 2024-04-25T12:57:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T12:57:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:12.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:57:12.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:57:12.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:12.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:12.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] boot 2024-04-25T12:57:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T12:57:12.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:12.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:57:12.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:57:12.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T12:57:12.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:12.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: osd.0 [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] boot 2024-04-25T12:57:12.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T12:57:12.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2269834501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4101c60 msgr2=0x7fa2c41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2269834501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4101c60 0x7fa2c41020e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009990 tx=0x7fa2b8009ca0 comp rx=0 tx=0).stop 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2269834501 shutdown_connections 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2269834501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c4102620 0x7fa2c4109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2269834501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103000 0x7fa2c4103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2269834501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4101c60 0x7fa2c41020e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2269834501 >> 172.21.15.104:0/2269834501 conn(0x7fa2c40fc710 msgr2=0x7fa2c40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2269834501 shutdown_connections 2024-04-25T12:57:13.816 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.816+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2269834501 wait complete. 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 Processor -- start 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 -- start start 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4101c60 0x7fa2c4105a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c4102620 0x7fa2c4104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 0x7fa2c4104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c410d650 con 0x7fa2c4101c60 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c410d4d0 con 0x7fa2c4103000 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c9cb6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c410d7d0 con 0x7fa2c4102620 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 0x7fa2c4104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 0x7fa2c4104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52988/0 (socket says 172.21.15.104:52988) 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.817+0000 7fa2c3fff700 1 -- 172.21.15.104:0/2412542485 learned_addr learned my addr 172.21.15.104:0/2412542485 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-25T12:57:13.817 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c2ffd700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c4102620 0x7fa2c4104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c37fe700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4101c60 0x7fa2c4105a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c3fff700 1 -- 172.21.15.104:0/2412542485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c4102620 msgr2=0x7fa2c4104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c3fff700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c4102620 0x7fa2c4104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c3fff700 1 -- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4101c60 msgr2=0x7fa2c4105a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c3fff700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4101c60 0x7fa2c4105a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c3fff700 1 -- 172.21.15.104:0/2412542485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c4104e40 con 0x7fa2c4103000 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c3fff700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 0x7fa2c4104590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00ada0 tx=0x7fa2bc00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc017020 con 0x7fa2c4103000 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa2bc00cda0 con 0x7fa2c4103000 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc005310 con 0x7fa2c4103000 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c41a2f40 con 0x7fa2c4103000 2024-04-25T12:57:13.818 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.818+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c41a3470 con 0x7fa2c4103000 2024-04-25T12:57:13.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.819+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c4069730 con 0x7fa2c4103000 2024-04-25T12:57:13.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.820+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa2bc00c8d0 con 0x7fa2c4103000 2024-04-25T12:57:13.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.821+0000 7fa2c0ff9700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa2ac0a3c30 0x7fa2ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:13.819 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.821+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa2bc0ce000 con 0x7fa2c4103000 2024-04-25T12:57:13.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.821+0000 7fa2c37fe700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa2ac0a3c30 0x7fa2ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:13.820 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.822+0000 7fa2c37fe700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa2ac0a3c30 0x7fa2ac0a60f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2c406af70 tx=0x7fa2b4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:13.823 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:13.826+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2bc021020 con 0x7fa2c4103000 2024-04-25T12:57:13.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi038 2024-04-25T12:57:13.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:13.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi038 to 17093M 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:13.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.028 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:14.030+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa2c40fe890 con 0x7fa2ac0a3c30 2024-04-25T12:57:14.028 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi038 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi038 to 17093M 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi038 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi038 to 17093M 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:14.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:14.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:14 smithi104 ceph-mon[32726]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:14.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:14 smithi104 ceph-mon[32726]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T12:57:14.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:14 smithi104 ceph-mon[32726]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:14.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:14.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:14.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:15.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:14 smithi038 ceph-mon[29400]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:15.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:14 smithi038 ceph-mon[29400]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T12:57:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:14 smithi038 ceph-mon[29400]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:15.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:15.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:14 smithi178 ceph-mon[33233]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:15.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:14 smithi178 ceph-mon[33233]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T12:57:15.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:14 smithi178 ceph-mon[33233]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:15.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:15.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:15.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:16 smithi178 ceph-mon[33233]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:16 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/1118042965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]: dispatch 2024-04-25T12:57:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:16 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]: dispatch 2024-04-25T12:57:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:16 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]': finished 2024-04-25T12:57:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:16 smithi178 ceph-mon[33233]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T12:57:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:16 smithi038 ceph-mon[29400]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:16 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/1118042965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]: dispatch 2024-04-25T12:57:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:16 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]: dispatch 2024-04-25T12:57:17.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:16 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]': finished 2024-04-25T12:57:17.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:16 smithi038 ceph-mon[29400]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T12:57:17.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:17.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:16 smithi104 ceph-mon[32726]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:16 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/1118042965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]: dispatch 2024-04-25T12:57:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:16 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]: dispatch 2024-04-25T12:57:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:16 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61eee122-04d2-413a-9203-29184e87e08f"}]': finished 2024-04-25T12:57:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:16 smithi104 ceph-mon[32726]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T12:57:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:18.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:17 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/2713257323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:17 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/2713257323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:18.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:17 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/2713257323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:19.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:18 smithi038 ceph-mon[29400]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:19.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:18 smithi104 ceph-mon[32726]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:19.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:18 smithi178 ceph-mon[33233]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:20.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:20 smithi104 ceph-mon[32726]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:21.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:20 smithi038 ceph-mon[29400]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:20 smithi178 ceph-mon[33233]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:23.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:22 smithi038 ceph-mon[29400]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:23.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:22 smithi104 ceph-mon[32726]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:23.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:22 smithi178 ceph-mon[33233]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:25.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:24 smithi038 ceph-mon[29400]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T12:57:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:24 smithi038 ceph-mon[29400]: Deploying daemon osd.1 on smithi104 2024-04-25T12:57:25.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:24 smithi104 ceph-mon[32726]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:25.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T12:57:25.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:25.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:24 smithi104 ceph-mon[32726]: Deploying daemon osd.1 on smithi104 2024-04-25T12:57:25.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:24 smithi178 ceph-mon[33233]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:25.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T12:57:25.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:25.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:24 smithi178 ceph-mon[33233]: Deploying daemon osd.1 on smithi104 2024-04-25T12:57:26.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:26 smithi104 ceph-mon[32726]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:27.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:26 smithi038 ceph-mon[29400]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:27.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:26 smithi178 ceph-mon[33233]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:29.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:28 smithi038 ceph-mon[29400]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:29.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:28 smithi104 ceph-mon[32726]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:29.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:28 smithi178 ceph-mon[33233]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:30.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:30.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:30.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:30.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:30.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:30.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:30.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:30.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:30.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:31.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:31 smithi104 ceph-mon[32726]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:31.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:31 smithi178 ceph-mon[33233]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:31.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:31 smithi038 ceph-mon[29400]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:32.728 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:32 smithi104 ceph-mon[32726]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:32.728 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:32.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:32.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:32.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:32.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:32.791 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.793+0000 7fa2c0ff9700 1 -- 172.21.15.104:0/2412542485 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2c40fe890 con 0x7fa2ac0a3c30 2024-04-25T12:57:32.792 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2024-04-25T12:57:32.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa2ac0a3c30 msgr2=0x7fa2ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:32.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa2ac0a3c30 0x7fa2ac0a60f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2c406af70 tx=0x7fa2b4009190 comp rx=0 tx=0).stop 2024-04-25T12:57:32.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 msgr2=0x7fa2c4104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:32.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 0x7fa2c4104590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00ada0 tx=0x7fa2bc00c330 comp rx=0 tx=0).stop 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 shutdown_connections 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa2ac0a3c30 0x7fa2ac0a60f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2c4103000 0x7fa2c4104590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4101c60 0x7fa2c4105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 --2- 172.21.15.104:0/2412542485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c4102620 0x7fa2c4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 >> 172.21.15.104:0/2412542485 conn(0x7fa2c40fc710 msgr2=0x7fa2c40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 shutdown_connections 2024-04-25T12:57:32.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:57:32.796+0000 7fa2c9cb6700 1 -- 172.21.15.104:0/2412542485 wait complete. 2024-04-25T12:57:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:32 smithi178 ceph-mon[33233]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:32 smithi038 ceph-mon[29400]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.423 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T12:57:33.426 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_4... 2024-04-25T12:57:33.426 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T12:57:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:57:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:33 smithi178 ceph-mon[33233]: from='osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:57:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:33 smithi178 ceph-mon[33233]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:57:33.764 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:33.764 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.764 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:33.764 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:33 smithi104 ceph-mon[32726]: from='osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:57:33.764 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:33 smithi104 ceph-mon[32726]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:57:34.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:34.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:34.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:33 smithi038 ceph-mon[29400]: from='osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:57:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:33 smithi038 ceph-mon[29400]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T12:57:34.975 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:34 smithi104 ceph-mon[32726]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:34.976 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:34 smithi104 ceph-mon[32726]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T12:57:34.976 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:34 smithi104 ceph-mon[32726]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T12:57:34.976 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:34 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:34.976 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:34 smithi104 ceph-mon[32726]: from='osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:57:34.976 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:34 smithi104 ceph-mon[32726]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:57:35.026 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:34 smithi178 ceph-mon[33233]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:35.026 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:34 smithi178 ceph-mon[33233]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T12:57:35.026 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:34 smithi178 ceph-mon[33233]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T12:57:35.026 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:34 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:35.026 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:34 smithi178 ceph-mon[33233]: from='osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:57:35.026 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:34 smithi178 ceph-mon[33233]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:57:35.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:34 smithi038 ceph-mon[29400]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:34 smithi038 ceph-mon[29400]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T12:57:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:34 smithi038 ceph-mon[29400]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T12:57:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:34 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:34 smithi038 ceph-mon[29400]: from='osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:57:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:34 smithi038 ceph-mon[29400]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:57:35.570 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T12:57:35.571 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T12:57:35.571 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-04-25T12:57:35.571 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-04-25T12:57:35.571 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.008406 s, 1.2 GB/s 2024-04-25T12:57:35.571 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-04-25T12:57:35.573 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1[37344]: 2024-04-25T12:57:35.227+0000 7f89c996d700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T12:57:35.573 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1[37344]: 2024-04-25T12:57:35.231+0000 7f89c1758700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:57:36.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:35 smithi038 ceph-mon[29400]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T12:57:36.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:35 smithi038 ceph-mon[29400]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T12:57:36.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:36.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:36.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:35 smithi038 ceph-mon[29400]: from='osd.1 ' entity='osd.1' 2024-04-25T12:57:36.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-mon[32726]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T12:57:36.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-mon[32726]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T12:57:36.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:36.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:36.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:35 smithi104 ceph-mon[32726]: from='osd.1 ' entity='osd.1' 2024-04-25T12:57:36.146 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:35 smithi178 ceph-mon[33233]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T12:57:36.146 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:35 smithi178 ceph-mon[33233]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T12:57:36.146 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:35 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:36.146 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:35 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:36.146 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:35 smithi178 ceph-mon[33233]: from='osd.1 ' entity='osd.1' 2024-04-25T12:57:36.174 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-04-25T12:57:36.410 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:57:37.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] boot 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] boot 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: osd.1 [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] boot 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:57:37.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:57:37.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:37.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:37.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:57:37.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 -- 172.21.15.178:0/2794155908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c102aa0 msgr2=0x7f494c102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2794155908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c102aa0 0x7f494c102ec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f493c009990 tx=0x7f493c009ca0 comp rx=0 tx=0).stop 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 -- 172.21.15.178:0/2794155908 shutdown_connections 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2794155908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f494c104640 0x7f494c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2794155908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c102aa0 0x7f494c102ec0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2794155908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f494c103ca0 0x7f494c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 -- 172.21.15.178:0/2794155908 >> 172.21.15.178:0/2794155908 conn(0x7f494c0fe040 msgr2=0x7f494c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:38.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 -- 172.21.15.178:0/2794155908 shutdown_connections 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.026+0000 7f4951fb2700 1 -- 172.21.15.178:0/2794155908 wait complete. 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 Processor -- start 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 -- start start 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f494c102aa0 0x7f494c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 0x7f494c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f494c104640 0x7f494c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f494c10e670 con 0x7f494c104640 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f494c10e4f0 con 0x7f494c102aa0 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4951fb2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f494c10e7f0 con 0x7f494c103ca0 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 0x7f494c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 0x7f494c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38918/0 (socket says 172.21.15.178:38918) 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 -- 172.21.15.178:0/2324979715 learned_addr learned my addr 172.21.15.178:0/2324979715 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:57:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f494b7fe700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f494c102aa0 0x7f494c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f494bfff700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f494c104640 0x7f494c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 -- 172.21.15.178:0/2324979715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f494c102aa0 msgr2=0x7f494c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f494c102aa0 0x7f494c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 -- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f494c104640 msgr2=0x7f494c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f494c104640 0x7f494c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.027+0000 7f4943fff700 1 -- 172.21.15.178:0/2324979715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c075c20 con 0x7f494c103ca0 2024-04-25T12:57:38.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.028+0000 7f4943fff700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 0x7f494c074e30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f493400c9b0 tx=0x7f493400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:38.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.028+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934007b00 con 0x7f494c103ca0 2024-04-25T12:57:38.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.028+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f493400ce50 con 0x7f494c103ca0 2024-04-25T12:57:38.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.028+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c1a2de0 con 0x7f494c103ca0 2024-04-25T12:57:38.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.028+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934018660 con 0x7f494c103ca0 2024-04-25T12:57:38.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.028+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c1a3240 con 0x7f494c103ca0 2024-04-25T12:57:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.029+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494c069730 con 0x7f494c103ca0 2024-04-25T12:57:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.030+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f493400f450 con 0x7f494c103ca0 2024-04-25T12:57:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.031+0000 7f49497fa700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49380a3bb0 0x7f49380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:57:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.031+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f49340cea70 con 0x7f494c103ca0 2024-04-25T12:57:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.031+0000 7f494b7fe700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49380a3bb0 0x7f49380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:57:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.031+0000 7f494b7fe700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49380a3bb0 0x7f49380a6070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f493c003870 tx=0x7f493c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:57:38.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.036+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4934014030 con 0x7f494c103ca0 2024-04-25T12:57:38.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:37 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi104 2024-04-25T12:57:38.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:37 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi104 to 17088M 2024-04-25T12:57:38.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:37 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi104 2024-04-25T12:57:38.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:37 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi104 to 17088M 2024-04-25T12:57:38.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:38.240+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f494c062e80 con 0x7f49380a3bb0 2024-04-25T12:57:38.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:37 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi104 2024-04-25T12:57:38.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:37 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi104 to 17088M 2024-04-25T12:57:39.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:38 smithi038 ceph-mon[29400]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:39.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:38 smithi038 ceph-mon[29400]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:57:39.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:39.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:39.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:39.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:38 smithi104 ceph-mon[32726]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:39.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:38 smithi104 ceph-mon[32726]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:57:39.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:39.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:39.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:39.150 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:38 smithi178 ceph-mon[33233]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:39.150 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:38 smithi178 ceph-mon[33233]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:57:39.150 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:57:39.150 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:57:39.150 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:40.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:39 smithi178 ceph-mon[33233]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:40.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:39 smithi038 ceph-mon[29400]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:40.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:39 smithi104 ceph-mon[32726]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:57:41.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:40 smithi038 ceph-mon[29400]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:41.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:40 smithi104 ceph-mon[32726]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:41.187 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:40 smithi178 ceph-mon[33233]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:42.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/3464174342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]: dispatch 2024-04-25T12:57:42.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:41 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]: dispatch 2024-04-25T12:57:42.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:41 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]': finished 2024-04-25T12:57:42.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:41 smithi038 ceph-mon[29400]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:57:42.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:57:42.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/3876055017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:42.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/3464174342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]: dispatch 2024-04-25T12:57:42.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:41 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]: dispatch 2024-04-25T12:57:42.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:41 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]': finished 2024-04-25T12:57:42.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:41 smithi104 ceph-mon[32726]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:57:42.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:57:42.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/3876055017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:42.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/3464174342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]: dispatch 2024-04-25T12:57:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:41 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]: dispatch 2024-04-25T12:57:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:41 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0d8bc3e-b869-46b9-b674-3f547e26ae93"}]': finished 2024-04-25T12:57:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:41 smithi178 ceph-mon[33233]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:57:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:57:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/3876055017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:57:43.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:42 smithi178 ceph-mon[33233]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:43.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:42 smithi038 ceph-mon[29400]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:43.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:42 smithi104 ceph-mon[32726]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:45.076 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:44 smithi178 ceph-mon[33233]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:45.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:44 smithi038 ceph-mon[29400]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:45.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:44 smithi104 ceph-mon[32726]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:46.881 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:46 smithi178 ceph-mon[33233]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:47.125 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:46 smithi104 ceph-mon[32726]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:47.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:46 smithi038 ceph-mon[29400]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:49.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:48 smithi178 ceph-mon[33233]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:57:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:48 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:48 smithi178 ceph-mon[33233]: Deploying daemon osd.2 on smithi178 2024-04-25T12:57:49.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:48 smithi038 ceph-mon[29400]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:49.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:57:49.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:48 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:48 smithi038 ceph-mon[29400]: Deploying daemon osd.2 on smithi178 2024-04-25T12:57:49.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:48 smithi104 ceph-mon[32726]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:49.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:57:49.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:48 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:57:49.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:48 smithi104 ceph-mon[32726]: Deploying daemon osd.2 on smithi178 2024-04-25T12:57:50.987 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:50 smithi178 ceph-mon[33233]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:51.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:50 smithi038 ceph-mon[29400]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:51.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:50 smithi104 ceph-mon[32726]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:52.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:52 smithi178 ceph-mon[33233]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:53.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:52 smithi038 ceph-mon[29400]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:53.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:52 smithi104 ceph-mon[32726]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:53.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:53.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:53.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:54.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:54.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:54.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:54.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:57:54.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:54.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:55.144 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:54 smithi178 ceph-mon[33233]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:55.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:54 smithi038 ceph-mon[29400]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:55.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:54 smithi104 ceph-mon[32726]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:56.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.707+0000 7f49497fa700 1 -- 172.21.15.178:0/2324979715 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f494c062e80 con 0x7f49380a3bb0 2024-04-25T12:57:56.709 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49380a3bb0 msgr2=0x7f49380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49380a3bb0 0x7f49380a6070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f493c003870 tx=0x7f493c005fd0 comp rx=0 tx=0).stop 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 msgr2=0x7f494c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 0x7f494c074e30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f493400c9b0 tx=0x7f493400ccc0 comp rx=0 tx=0).stop 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 shutdown_connections 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49380a3bb0 0x7f49380a6070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f494c102aa0 0x7f494c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:56.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f494c104640 0x7f494c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:56.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 --2- 172.21.15.178:0/2324979715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f494c103ca0 0x7f494c074e30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:57:56.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 >> 172.21.15.178:0/2324979715 conn(0x7f494c0fe040 msgr2=0x7f494c106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:57:56.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 shutdown_connections 2024-04-25T12:57:56.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:57:56.710+0000 7f4951fb2700 1 -- 172.21.15.178:0/2324979715 wait complete. 2024-04-25T12:57:57.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:56 smithi178 ceph-mon[33233]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:57.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:56 smithi038 ceph-mon[29400]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:57.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:56 smithi104 ceph-mon[32726]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:57.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:57:57.332 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T12:57:57.334 INFO:tasks.cephadm:Deploying osd.3 on smithi038 with /dev/vg_nvme/lv_3... 2024-04-25T12:57:57.335 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:57:57.586 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:57:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:57 smithi104 ceph-mon[32726]: from='osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:57:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:57 smithi104 ceph-mon[32726]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:57:58.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:57 smithi178 ceph-mon[33233]: from='osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:57:58.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:57 smithi178 ceph-mon[33233]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:57:58.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:57 smithi038 ceph-mon[29400]: from='osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:57:58.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:57 smithi038 ceph-mon[29400]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:57:58.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-mon[33233]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:58.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-mon[33233]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:57:58.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-mon[33233]: from='osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:57:58.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-mon[33233]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:57:58.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:57:58.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-mon[33233]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:57:58.989 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2[38383]: 2024-04-25T12:57:58.930+0000 7fa9febe8700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T12:57:58.989 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:57:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2[38383]: 2024-04-25T12:57:58.934+0000 7fa9fa9db700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:57:59.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:58 smithi038 ceph-mon[29400]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:59.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:58 smithi038 ceph-mon[29400]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:57:59.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:58 smithi038 ceph-mon[29400]: from='osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:57:59.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:58 smithi038 ceph-mon[29400]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:57:59.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:57:59.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:58 smithi038 ceph-mon[29400]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:57:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:58 smithi104 ceph-mon[32726]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:57:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:58 smithi104 ceph-mon[32726]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:57:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:58 smithi104 ceph-mon[32726]: from='osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:57:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:58 smithi104 ceph-mon[32726]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:57:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:57:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:58 smithi104 ceph-mon[32726]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:57:59.721 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:57:59.721 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:57:59.722 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-25T12:57:59.722 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-25T12:57:59.722 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.015641 s, 670 MB/s 2024-04-25T12:57:59.722 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-25T12:58:00.134 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:59 smithi178 ceph-mon[33233]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T12:58:00.134 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:59 smithi178 ceph-mon[33233]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:58:00.134 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:59 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:00.134 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:57:59 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:00.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:59 smithi038 ceph-mon[29400]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T12:58:00.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:59 smithi038 ceph-mon[29400]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:58:00.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:59 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:00.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:57:59 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:00.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:59 smithi104 ceph-mon[32726]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T12:58:00.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:59 smithi104 ceph-mon[32726]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:58:00.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:59 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:00.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:57:59 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:00.413 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-04-25T12:58:00.656 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:58:01.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] boot 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:01.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:01.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:01.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:00 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:58:01.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:58:01.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:58:01.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] boot 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:01.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:00 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:58:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:58:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:58:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: osd.2 [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] boot 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:00 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:02.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:01 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi178 2024-04-25T12:58:02.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:01 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi178 to 17088M 2024-04-25T12:58:02.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:01 smithi038 ceph-mon[29400]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:58:02.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:01 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:58:02.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:01 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi178 2024-04-25T12:58:02.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:01 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi178 to 17088M 2024-04-25T12:58:02.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:01 smithi178 ceph-mon[33233]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:58:02.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:01 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:58:02.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:01 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi178 2024-04-25T12:58:02.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:01 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi178 to 17088M 2024-04-25T12:58:02.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:01 smithi104 ceph-mon[32726]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:58:02.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:01 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 -- 172.21.15.38:0/1477926155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec103cc0 msgr2=0x7f46ec104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 --2- 172.21.15.38:0/1477926155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec103cc0 0x7f46ec104120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46e00099c0 tx=0x7f46e0009cd0 comp rx=0 tx=0).stop 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 -- 172.21.15.38:0/1477926155 shutdown_connections 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 --2- 172.21.15.38:0/1477926155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec102ac0 0x7f46ec102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 --2- 172.21.15.38:0/1477926155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46ec104660 0x7f46ec10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 --2- 172.21.15.38:0/1477926155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec103cc0 0x7f46ec104120 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 -- 172.21.15.38:0/1477926155 >> 172.21.15.38:0/1477926155 conn(0x7f46ec0fe060 msgr2=0x7f46ec1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 -- 172.21.15.38:0/1477926155 shutdown_connections 2024-04-25T12:58:02.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 -- 172.21.15.38:0/1477926155 wait complete. 2024-04-25T12:58:02.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.330+0000 7f46f1998700 1 Processor -- start 2024-04-25T12:58:02.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 -- start start 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46ec102ac0 0x7f46ec19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec103cc0 0x7f46ec19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 0x7f46ec1a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10e670 con 0x7f46ec104660 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10e4f0 con 0x7f46ec103cc0 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46f1998700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10e7f0 con 0x7f46ec102ac0 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 0x7f46ec1a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 0x7f46ec1a2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43972/0 (socket says 172.21.15.38:43972) 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 -- 172.21.15.38:0/2138055519 learned_addr learned my addr 172.21.15.38:0/2138055519 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eaffd700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46ec102ac0 0x7f46ec19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46ea7fc700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec103cc0 0x7f46ec19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:02.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 -- 172.21.15.38:0/2138055519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46ec102ac0 msgr2=0x7f46ec19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46ec102ac0 0x7f46ec19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 -- 172.21.15.38:0/2138055519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec103cc0 msgr2=0x7f46ec19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec103cc0 0x7f46ec19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 -- 172.21.15.38:0/2138055519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1a2b20 con 0x7f46ec104660 2024-04-25T12:58:02.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46ea7fc700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec103cc0 0x7f46ec19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.331+0000 7f46eb7fe700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 0x7f46ec1a2480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46e4007f10 tx=0x7f46e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.332+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4004bb0 con 0x7f46ec104660 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.332+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f46e4004d50 con 0x7f46ec104660 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.332+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40133f0 con 0x7f46ec104660 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.332+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1a2e10 con 0x7f46ec104660 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.332+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a3270 con 0x7f46ec104660 2024-04-25T12:58:02.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.333+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec069730 con 0x7f46ec104660 2024-04-25T12:58:02.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.334+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f46e401a020 con 0x7f46ec104660 2024-04-25T12:58:02.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.335+0000 7f46f0996700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f46d40a3bb0 0x7f46d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:02.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.335+0000 7f46eaffd700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f46d40a3bb0 0x7f46d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:02.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.335+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f46e40cecc0 con 0x7f46ec104660 2024-04-25T12:58:02.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.335+0000 7f46eaffd700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f46d40a3bb0 0x7f46d40a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f46ec103b20 tx=0x7f46dc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:02.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.339+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46e4010070 con 0x7f46ec104660 2024-04-25T12:58:02.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:02.555+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f46ec076390 con 0x7f46d40a3bb0 2024-04-25T12:58:03.204 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:58:03.204 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:03.204 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:58:03.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:58:03.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:58:03.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:03.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:03.205 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45881]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45881]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45881]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45881]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45893]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45893]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45893]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.209 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45893]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:03.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:03.323 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41476]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41476]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41476]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41476]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41479]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.324 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41479]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.488 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.488 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42297]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.488 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42297]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.489 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:58:03.489 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.489 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42300]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.489 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42300]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42303]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42303]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42306]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 sudo[42306]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45908]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45908]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45911]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 sudo[45911]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:58:03.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41482]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:58:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:58:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41485]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:58:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 sudo[41485]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:58:04.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:04.210+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46e4098ad0 con 0x7f46ec104660 2024-04-25T12:58:04.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:04.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:03 smithi178 ceph-mon[33233]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.252 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.253 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:03 smithi038 ceph-mon[29400]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:04.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:58:04.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:58:04.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:58:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:58:04.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:03 smithi104 ceph-mon[32726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:58:05.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:04 smithi178 ceph-mon[33233]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:05.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:04 smithi178 ceph-mon[33233]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T12:58:05.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:04 smithi178 ceph-mon[33233]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T12:58:05.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:04 smithi038 ceph-mon[29400]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:05.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:04 smithi038 ceph-mon[29400]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T12:58:05.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:04 smithi038 ceph-mon[29400]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T12:58:05.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:04 smithi104 ceph-mon[32726]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:05.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:04 smithi104 ceph-mon[32726]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T12:58:05.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:04 smithi104 ceph-mon[32726]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T12:58:06.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:05 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/817523735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a"}]: dispatch 2024-04-25T12:58:06.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:05 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/817523735' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a"}]': finished 2024-04-25T12:58:06.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:05 smithi178 ceph-mon[33233]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:58:06.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:05 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:06.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:05 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/817523735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a"}]: dispatch 2024-04-25T12:58:06.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:05 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/817523735' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a"}]': finished 2024-04-25T12:58:06.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:05 smithi038 ceph-mon[29400]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:58:06.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:05 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:06.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:05 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/817523735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a"}]: dispatch 2024-04-25T12:58:06.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:05 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/817523735' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a"}]': finished 2024-04-25T12:58:06.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:05 smithi104 ceph-mon[32726]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:58:06.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:05 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:07.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:06 smithi178 ceph-mon[33233]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:07.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:06 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3686032808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:07.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:06 smithi038 ceph-mon[29400]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:07.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:06 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3686032808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:07.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:06 smithi104 ceph-mon[32726]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:07.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:06 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3686032808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:09.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:08 smithi178 ceph-mon[33233]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:09.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:08 smithi038 ceph-mon[29400]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:09.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:08 smithi104 ceph-mon[32726]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:11.167 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:10 smithi038 ceph-mon[29400]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:11.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:10 smithi104 ceph-mon[32726]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:11.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:10 smithi178 ceph-mon[33233]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:13.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:12 smithi038 ceph-mon[29400]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:13.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:12 smithi104 ceph-mon[32726]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:13.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:12 smithi178 ceph-mon[33233]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:14.307 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:58:14.307 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:14.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:58:14.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:14.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:58:14.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:15.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:15 smithi038 ceph-mon[29400]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:15.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:15 smithi038 ceph-mon[29400]: Deploying daemon osd.3 on smithi038 2024-04-25T12:58:15.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:15 smithi104 ceph-mon[32726]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:15.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:15 smithi104 ceph-mon[32726]: Deploying daemon osd.3 on smithi038 2024-04-25T12:58:15.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:15 smithi178 ceph-mon[33233]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:15.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:15 smithi178 ceph-mon[33233]: Deploying daemon osd.3 on smithi038 2024-04-25T12:58:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:17 smithi178 ceph-mon[33233]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:17.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:17 smithi038 ceph-mon[29400]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:17.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:17 smithi104 ceph-mon[32726]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:19.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:19 smithi038 ceph-mon[29400]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:19.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:19 smithi104 ceph-mon[32726]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:19.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:19 smithi178 ceph-mon[33233]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:20.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:20 smithi038 ceph-mon[29400]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:20.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:20 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:20.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:20 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:20.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:20 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:20.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:20 smithi104 ceph-mon[32726]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:20.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:20 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:20.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:20 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:20.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:20 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:20.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:20 smithi178 ceph-mon[33233]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:20.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:20 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:20.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:20 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:20.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:20 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:22.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:22 smithi038 ceph-mon[29400]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:22.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:22 smithi104 ceph-mon[32726]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:22.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:22 smithi178 ceph-mon[33233]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:23.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.422+0000 7f46f0996700 1 -- 172.21.15.38:0/2138055519 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46ec076390 con 0x7f46d40a3bb0 2024-04-25T12:58:23.420 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 3 on host 'smithi038' 2024-04-25T12:58:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f46d40a3bb0 msgr2=0x7f46d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f46d40a3bb0 0x7f46d40a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f46ec103b20 tx=0x7f46dc007400 comp rx=0 tx=0).stop 2024-04-25T12:58:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 msgr2=0x7f46ec1a2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 0x7f46ec1a2480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46e4007f10 tx=0x7f46e400d3b0 comp rx=0 tx=0).stop 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 shutdown_connections 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f46d40a3bb0 0x7f46d40a6070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46ec104660 0x7f46ec1a2480 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46ec102ac0 0x7f46ec19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 --2- 172.21.15.38:0/2138055519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec103cc0 0x7f46ec19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.424+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 >> 172.21.15.38:0/2138055519 conn(0x7f46ec0fe060 msgr2=0x7f46ec074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.425+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 shutdown_connections 2024-04-25T12:58:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:58:23.425+0000 7f46f1998700 1 -- 172.21.15.38:0/2138055519 wait complete. 2024-04-25T12:58:23.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:23.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:23.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:58:23.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.790 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:23.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:24.140 DEBUG:teuthology.orchestra.run.smithi038:osd.3> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T12:58:24.143 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-04-25T12:58:24.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:58:24.385 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:58:24.711 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-mon[29400]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:24.711 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-mon[29400]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:58:24.711 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-mon[29400]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:58:24.711 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-mon[29400]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:58:24.711 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:24.711 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3[48060]: 2024-04-25T12:58:24.482+0000 7f0b40c8b700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T12:58:24.712 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 12:58:24 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3[48060]: 2024-04-25T12:58:24.487+0000 7f0b3ca7e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:58:24.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:24 smithi178 ceph-mon[33233]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:24.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:24 smithi178 ceph-mon[33233]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:58:24.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:24 smithi178 ceph-mon[33233]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:58:24.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:24 smithi178 ceph-mon[33233]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:58:24.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:24.799 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:24 smithi104 ceph-mon[32726]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:24.799 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:24 smithi104 ceph-mon[32726]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:58:24.799 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:24 smithi104 ceph-mon[32726]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:58:24.799 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:24 smithi104 ceph-mon[32726]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:58:24.799 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:25.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:25 smithi038 ceph-mon[29400]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:58:25.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:25 smithi038 ceph-mon[29400]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T12:58:25.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:25.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:25 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:25.757 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:25 smithi104 ceph-mon[32726]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:58:25.758 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:25 smithi104 ceph-mon[32726]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T12:58:25.758 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:25 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:25.758 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:25 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:25.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:25 smithi178 ceph-mon[33233]: from='osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:58:25.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:25 smithi178 ceph-mon[33233]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T12:58:25.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:25.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:25 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:58:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:58:26.344 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-25T12:58:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-25T12:58:26.344 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115781 s, 906 MB/s 2024-04-25T12:58:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] boot 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:26.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:26 smithi104 ceph-mon[32726]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:58:26.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:58:26.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:58:26.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:26.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] boot 2024-04-25T12:58:26.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:58:26.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:26.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:26 smithi038 ceph-mon[29400]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:58:26.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: osd.3 [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] boot 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:58:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:26 smithi178 ceph-mon[33233]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:58:27.212 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:58:27.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:28.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 -- 172.21.15.104:0/2083212913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 msgr2=0x7f4520101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 --2- 172.21.15.104:0/2083212913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f4520101dc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f45100099c0 tx=0x7f4510009cd0 comp rx=0 tx=0).stop 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 -- 172.21.15.104:0/2083212913 shutdown_connections 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 --2- 172.21.15.104:0/2083212913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f4520101dc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 --2- 172.21.15.104:0/2083212913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4520103540 0x7f4520109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 --2- 172.21.15.104:0/2083212913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4520102ba0 0x7f4520103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 -- 172.21.15.104:0/2083212913 >> 172.21.15.104:0/2083212913 conn(0x7f45200fcf60 msgr2=0x7f45200ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 -- 172.21.15.104:0/2083212913 shutdown_connections 2024-04-25T12:58:28.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.890+0000 7f4525395700 1 -- 172.21.15.104:0/2083212913 wait complete. 2024-04-25T12:58:28.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.891+0000 7f4525395700 1 Processor -- start 2024-04-25T12:58:28.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.891+0000 7f4525395700 1 -- start start 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.891+0000 7f4525395700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f45201058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.891+0000 7f4525395700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4520102ba0 0x7f4520103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4525395700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4520103540 0x7f45201044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4525395700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f452010d5d0 con 0x7f4520103540 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4525395700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f452010d450 con 0x7f45201019a0 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4525395700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f452010d750 con 0x7f4520102ba0 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f45201058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f45201058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:60992/0 (socket says 172.21.15.104:60992) 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 -- 172.21.15.104:0/2134897021 learned_addr learned my addr 172.21.15.104:0/2134897021 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451f7fe700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4520103540 0x7f45201044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451e7fc700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4520102ba0 0x7f4520103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 -- 172.21.15.104:0/2134897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4520102ba0 msgr2=0x7f4520103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4520102ba0 0x7f4520103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:28.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 -- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4520103540 msgr2=0x7f45201044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4520103540 0x7f45201044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 -- 172.21.15.104:0/2134897021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4520104d90 con 0x7f45201019a0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f451effd700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f45201058e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f451000b580 tx=0x7f4510009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510027070 con 0x7f45201019a0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f45100058e0 con 0x7f45201019a0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45201a2e80 con 0x7f45201019a0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510022d40 con 0x7f45201019a0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.892+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a3220 con 0x7f45201019a0 2024-04-25T12:58:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.893+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4520069760 con 0x7f45201019a0 2024-04-25T12:58:28.893 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.894+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f451002a070 con 0x7f45201019a0 2024-04-25T12:58:28.893 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.895+0000 7f4507fff700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f45080a3c40 0x7f45080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:28.893 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.895+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f45100d02f0 con 0x7f45201019a0 2024-04-25T12:58:28.893 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.895+0000 7f451e7fc700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f45080a3c40 0x7f45080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:28.893 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.895+0000 7f451e7fc700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f45080a3c40 0x7f45080a6100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4514006fd0 tx=0x7f4514008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:28.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:28.899+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4520069760 con 0x7f45201019a0 2024-04-25T12:58:28.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:28 smithi178 ceph-mon[33233]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-04-25T12:58:28.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:28 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi038 2024-04-25T12:58:28.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:28 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi038 to 8546M 2024-04-25T12:58:28.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:28 smithi178 ceph-mon[33233]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:58:29.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:28 smithi038 ceph-mon[29400]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-04-25T12:58:29.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:28 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi038 2024-04-25T12:58:29.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:28 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi038 to 8546M 2024-04-25T12:58:29.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:28 smithi038 ceph-mon[29400]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:58:29.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:28 smithi104 ceph-mon[32726]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-04-25T12:58:29.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:28 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi038 2024-04-25T12:58:29.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:28 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi038 to 8546M 2024-04-25T12:58:29.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:28 smithi104 ceph-mon[32726]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:58:29.106 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:29.109+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f45200ff150 con 0x7f45080a3c40 2024-04-25T12:58:29.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:29 smithi104 ceph-mon[32726]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T12:58:29.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:29 smithi104 ceph-mon[32726]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:29.938 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:29.938 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:29.938 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:29.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:29 smithi178 ceph-mon[33233]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T12:58:29.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:29 smithi178 ceph-mon[33233]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:29.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:30.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:29 smithi038 ceph-mon[29400]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T12:58:30.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:29 smithi038 ceph-mon[29400]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:30.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:30.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:30.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:30.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:30 smithi104 ceph-mon[32726]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-04-25T12:58:30.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:30 smithi178 ceph-mon[33233]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-04-25T12:58:31.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:30 smithi038 ceph-mon[29400]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-04-25T12:58:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T12:58:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/563505307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]: dispatch 2024-04-25T12:58:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]: dispatch 2024-04-25T12:58:32.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]': finished 2024-04-25T12:58:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:58:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/3640203236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T12:58:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/563505307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]: dispatch 2024-04-25T12:58:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]: dispatch 2024-04-25T12:58:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]': finished 2024-04-25T12:58:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:58:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/3640203236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:33.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T12:58:33.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/563505307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]: dispatch 2024-04-25T12:58:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]: dispatch 2024-04-25T12:58:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e42ba42a-a706-42ba-84d0-85ef48660c60"}]': finished 2024-04-25T12:58:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:58:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/3640203236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:34.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:34 smithi178 ceph-mon[33233]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:35.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:34 smithi038 ceph-mon[29400]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:35.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:34 smithi104 ceph-mon[32726]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:36.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:36 smithi178 ceph-mon[33233]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:37.040 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:36 smithi104 ceph-mon[32726]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:37.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:36 smithi038 ceph-mon[29400]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:38.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:38 smithi104 ceph-mon[32726]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:38.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:38 smithi178 ceph-mon[33233]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:39.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:38 smithi038 ceph-mon[29400]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:39.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:39 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:58:39.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:39 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:39.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:39 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:58:39.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:39 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:40.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:39 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:58:40.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:39 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:40.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:40 smithi104 ceph-mon[32726]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:40.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:40 smithi104 ceph-mon[32726]: Deploying daemon osd.4 on smithi104 2024-04-25T12:58:40.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:40 smithi178 ceph-mon[33233]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:40.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:40 smithi178 ceph-mon[33233]: Deploying daemon osd.4 on smithi104 2024-04-25T12:58:41.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:40 smithi038 ceph-mon[29400]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:41.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:40 smithi038 ceph-mon[29400]: Deploying daemon osd.4 on smithi104 2024-04-25T12:58:42.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:42 smithi178 ceph-mon[33233]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:42.995 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:42 smithi104 ceph-mon[32726]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:43.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:42 smithi038 ceph-mon[29400]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:44.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:44 smithi104 ceph-mon[32726]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:44.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:44 smithi178 ceph-mon[33233]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:45.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:44 smithi038 ceph-mon[29400]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:46.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:46.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:46.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:47.211 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:47 smithi104 ceph-mon[32726]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:47.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:47 smithi038 ceph-mon[29400]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:47.389 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:47 smithi178 ceph-mon[33233]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:48.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.978+0000 7f4507fff700 1 -- 172.21.15.104:0/2134897021 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45200ff150 con 0x7f45080a3c40 2024-04-25T12:58:48.977 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f45080a3c40 msgr2=0x7f45080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f45080a3c40 0x7f45080a6100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4514006fd0 tx=0x7f4514008040 comp rx=0 tx=0).stop 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 msgr2=0x7f45201058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f45201058e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f451000b580 tx=0x7f4510009580 comp rx=0 tx=0).stop 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 shutdown_connections 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f45080a3c40 0x7f45080a6100 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45201019a0 0x7f45201058e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:48.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4520103540 0x7f45201044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:48.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 --2- 172.21.15.104:0/2134897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4520102ba0 0x7f4520103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:48.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.981+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 >> 172.21.15.104:0/2134897021 conn(0x7f45200fcf60 msgr2=0x7f45200fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:48.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.982+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 shutdown_connections 2024-04-25T12:58:48.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:58:48.982+0000 7f4525395700 1 -- 172.21.15.104:0/2134897021 wait complete. 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:58:49.289 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:58:49.290 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.290 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.290 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:49 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:49.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:49 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:49.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:58:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:49 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:58:49.652 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T12:58:49.655 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-04-25T12:58:49.655 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:58:49.901 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:58:50.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-mon[32726]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:58:50.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-mon[32726]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:58:50.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:50.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-mon[32726]: from='osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:58:50.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-mon[32726]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:58:50.323 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T12:58:50.105+0000 7fcecb092700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T12:58:50.323 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 12:58:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T12:58:50.113+0000 7fcec6e85700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:58:50.357 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:50 smithi178 ceph-mon[33233]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:58:50.357 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:50 smithi178 ceph-mon[33233]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:58:50.357 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:50.357 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:50 smithi178 ceph-mon[33233]: from='osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:58:50.357 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:50 smithi178 ceph-mon[33233]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:58:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:50 smithi038 ceph-mon[29400]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:58:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:50 smithi038 ceph-mon[29400]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:58:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:50 smithi038 ceph-mon[29400]: from='osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:58:50.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:50 smithi038 ceph-mon[29400]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T12:58:51.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:51 smithi178 ceph-mon[33233]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:51.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:51 smithi178 ceph-mon[33233]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T12:58:51.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:51 smithi178 ceph-mon[33233]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T12:58:51.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:51 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:51.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:51 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:51.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:51 smithi038 ceph-mon[29400]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:51.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:51 smithi038 ceph-mon[29400]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T12:58:51.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:51 smithi038 ceph-mon[29400]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T12:58:51.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:51.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:51.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:51 smithi104 ceph-mon[32726]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:51.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:51 smithi104 ceph-mon[32726]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T12:58:51.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:51 smithi104 ceph-mon[32726]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T12:58:51.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:51 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:51.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:51 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00873702 s, 1.2 GB/s 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 2024-04-25T12:58:51.823 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-04-25T12:58:52.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:52 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:58:52.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:52 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:58:52.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:52 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:52 smithi104 ceph-mon[32726]: osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] boot 2024-04-25T12:58:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:52 smithi104 ceph-mon[32726]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:58:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:52 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:52.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:52 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:58:52.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:52 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:58:52.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:52 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:52.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:52 smithi178 ceph-mon[33233]: osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] boot 2024-04-25T12:58:52.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:52 smithi178 ceph-mon[33233]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:58:52.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:52 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:52.571 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-04-25T12:58:52.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:52 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:58:52.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:52 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:58:52.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:52 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:52.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:52 smithi038 ceph-mon[29400]: osd.4 [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] boot 2024-04-25T12:58:52.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:52 smithi038 ceph-mon[29400]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:58:52.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:52 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:58:52.815 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:58:53.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:53.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T12:58:53.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:53.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T12:58:53.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:53.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:58:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:58:54.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:54 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi104 2024-04-25T12:58:54.434 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:54 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi104 to 8544M 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 -- 172.21.15.178:0/3563552060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc103d30 msgr2=0x7fc3bc104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 --2- 172.21.15.178:0/3563552060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc103d30 0x7fc3bc104190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009990 tx=0x7fc3a4009ca0 comp rx=0 tx=0).stop 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 -- 172.21.15.178:0/3563552060 shutdown_connections 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 --2- 172.21.15.178:0/3563552060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3bc100d50 0x7fc3bc101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 --2- 172.21.15.178:0/3563552060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 --2- 172.21.15.178:0/3563552060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc103d30 0x7fc3bc104190 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 -- 172.21.15.178:0/3563552060 >> 172.21.15.178:0/3563552060 conn(0x7fc3bc0fc6d0 msgr2=0x7fc3bc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 -- 172.21.15.178:0/3563552060 shutdown_connections 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 -- 172.21.15.178:0/3563552060 wait complete. 2024-04-25T12:58:54.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.424+0000 7fc3c1731700 1 Processor -- start 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 -- start start 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3bc100d50 0x7fc3bc103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3bc103d30 0x7fc3bc101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc10e630 con 0x7fc3bc103d30 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc10e4b0 con 0x7fc3bc100d50 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3c1731700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc10e7b0 con 0x7fc3bc1046d0 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34604/0 (socket says 172.21.15.178:34604) 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 -- 172.21.15.178:0/2980300388 learned_addr learned my addr 172.21.15.178:0/2980300388 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3baffd700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3bc100d50 0x7fc3bc103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:54.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3b27fc700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3bc103d30 0x7fc3bc101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 -- 172.21.15.178:0/2980300388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3bc100d50 msgr2=0x7fc3bc103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3bc100d50 0x7fc3bc103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 -- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3bc103d30 msgr2=0x7fc3bc101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3bc103d30 0x7fc3bc101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 -- 172.21.15.178:0/2980300388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc102ad0 con 0x7fc3bc1046d0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.425+0000 7fc3bb7fe700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc102220 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400d980 tx=0x7fc3b400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.426+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4017070 con 0x7fc3bc1046d0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.426+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc3b4005020 con 0x7fc3bc1046d0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.426+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc1a2e00 con 0x7fc3bc1046d0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.426+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b400fb30 con 0x7fc3bc1046d0 2024-04-25T12:58:54.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.426+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc1a3260 con 0x7fc3bc1046d0 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.427+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc069730 con 0x7fc3bc1046d0 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.428+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3b40051d0 con 0x7fc3bc1046d0 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.428+0000 7fc3b8ff9700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3a80a3c40 0x7fc3a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.428+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fc3b40cf130 con 0x7fc3bc1046d0 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.429+0000 7fc3baffd700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3a80a3c40 0x7fc3a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.429+0000 7fc3baffd700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3a80a3c40 0x7fc3a80a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc06a520 tx=0x7fc3ac0091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:58:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.433+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3b4020020 con 0x7fc3bc1046d0 2024-04-25T12:58:54.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:54 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi104 2024-04-25T12:58:54.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:54 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi104 to 8544M 2024-04-25T12:58:54.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:54 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi104 2024-04-25T12:58:54.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:54 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi104 to 8544M 2024-04-25T12:58:54.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:58:54.638+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3bc062e80 con 0x7fc3a80a3c40 2024-04-25T12:58:55.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:55 smithi178 ceph-mon[33233]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:55.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:55 smithi178 ceph-mon[33233]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:55.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:55 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:55.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:55 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:55.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:55 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:55.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:55 smithi038 ceph-mon[29400]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:55.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:55 smithi038 ceph-mon[29400]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:55.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:55 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:55.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:55 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:55.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:55 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:55.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:55 smithi104 ceph-mon[32726]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:55.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:55 smithi104 ceph-mon[32726]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:58:55.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:55 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:58:55.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:55 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:58:55.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:55 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:58:56.413 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:56 smithi178 ceph-mon[33233]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:56.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:56 smithi038 ceph-mon[29400]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:56.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:56 smithi104 ceph-mon[32726]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:58.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:58.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/3796065059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]: dispatch 2024-04-25T12:58:58.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]: dispatch 2024-04-25T12:58:58.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]': finished 2024-04-25T12:58:58.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:58:58.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:58:58.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:58:58 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/1782005063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:58.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:58.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/3796065059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]: dispatch 2024-04-25T12:58:58.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]: dispatch 2024-04-25T12:58:58.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]': finished 2024-04-25T12:58:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:58:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:58:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:58:58 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/1782005063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:58:58.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:58:58.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/3796065059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]: dispatch 2024-04-25T12:58:58.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]: dispatch 2024-04-25T12:58:58.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93a59196-6a9a-4233-ad44-dee831b56089"}]': finished 2024-04-25T12:58:58.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:58:58.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:58:58.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:58:58 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/1782005063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:00.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:00 smithi038 ceph-mon[29400]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:00.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:00 smithi104 ceph-mon[32726]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:00.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:00 smithi178 ceph-mon[33233]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:02.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:02 smithi038 ceph-mon[29400]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:02.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:02 smithi178 ceph-mon[33233]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:02.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:02 smithi104 ceph-mon[32726]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:04.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:04 smithi038 ceph-mon[29400]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:04.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:04 smithi104 ceph-mon[32726]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:04 smithi178 ceph-mon[33233]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:05.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:05 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:59:05.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:05 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:05.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:05 smithi178 ceph-mon[33233]: Deploying daemon osd.5 on smithi178 2024-04-25T12:59:05.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:05 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:59:05.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:05 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:05.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:05 smithi038 ceph-mon[29400]: Deploying daemon osd.5 on smithi178 2024-04-25T12:59:05.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:05 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:59:05.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:05 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:05.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:05 smithi104 ceph-mon[32726]: Deploying daemon osd.5 on smithi178 2024-04-25T12:59:06.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:06 smithi178 ceph-mon[33233]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:06.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:06 smithi038 ceph-mon[29400]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:06.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:06 smithi104 ceph-mon[32726]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:08.536 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:08 smithi178 ceph-mon[33233]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:08.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:08 smithi038 ceph-mon[29400]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:08.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:08 smithi104 ceph-mon[32726]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:10.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:10 smithi178 ceph-mon[33233]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:10.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:10.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:10.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:10.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:10 smithi038 ceph-mon[29400]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:10.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:10.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:10.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:10.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:10 smithi104 ceph-mon[32726]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:12.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:12 smithi178 ceph-mon[33233]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:12.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:12 smithi038 ceph-mon[29400]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:12.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:12 smithi104 ceph-mon[32726]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:13.711 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-04-25T12:59:13.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.713+0000 7fc3b8ff9700 1 -- 172.21.15.178:0/2980300388 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3bc062e80 con 0x7fc3a80a3c40 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3a80a3c40 msgr2=0x7fc3a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3a80a3c40 0x7fc3a80a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc06a520 tx=0x7fc3ac0091c0 comp rx=0 tx=0).stop 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 msgr2=0x7fc3bc102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc102220 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400d980 tx=0x7fc3b400dc90 comp rx=0 tx=0).stop 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 shutdown_connections 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3a80a3c40 0x7fc3a80a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3bc100d50 0x7fc3bc103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3bc1046d0 0x7fc3bc102220 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:13.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 --2- 172.21.15.178:0/2980300388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3bc103d30 0x7fc3bc101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:13.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 >> 172.21.15.178:0/2980300388 conn(0x7fc3bc0fc6d0 msgr2=0x7fc3bc107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:13.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 shutdown_connections 2024-04-25T12:59:13.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T12:59:13.715+0000 7fc3c1731700 1 -- 172.21.15.178:0/2980300388 wait complete. 2024-04-25T12:59:14.264 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T12:59:14.266 INFO:tasks.cephadm:Deploying osd.6 on smithi038 with /dev/vg_nvme/lv_2... 2024-04-25T12:59:14.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:59:14.524 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:59:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: from='osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:59:14.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:14 smithi178 ceph-mon[33233]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:59:14.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:14.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:14.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: from='osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:59:14.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:14 smithi038 ceph-mon[29400]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:59:15.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:15.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:15.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:15.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:15.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:15.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: from='osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:59:15.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:14 smithi104 ceph-mon[32726]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:59:15.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:15 smithi178 ceph-mon[33233]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:59:15.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:15 smithi178 ceph-mon[33233]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:59:15.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:15 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:15.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:15 smithi178 ceph-mon[33233]: from='osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:59:15.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:15 smithi178 ceph-mon[33233]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:59:16.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:15 smithi038 ceph-mon[29400]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:59:16.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:15 smithi038 ceph-mon[29400]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:59:16.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:15 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:16.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:15 smithi038 ceph-mon[29400]: from='osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:59:16.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:15 smithi038 ceph-mon[29400]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:59:16.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:15 smithi104 ceph-mon[32726]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:59:16.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:15 smithi104 ceph-mon[32726]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:59:16.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:15 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:16.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:15 smithi104 ceph-mon[32726]: from='osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:59:16.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:15 smithi104 ceph-mon[32726]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T12:59:16.489 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T12:59:16.236+0000 7f4220a1c700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T12:59:16.489 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T12:59:16.245+0000 7f4218807700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:59:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:59:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:59:16.850 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-25T12:59:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-25T12:59:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142102 s, 738 MB/s 2024-04-25T12:59:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-25T12:59:16.898 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-mon[33233]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:16.898 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-mon[33233]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T12:59:16.898 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-mon[33233]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T12:59:16.898 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:16.898 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:16.898 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:16 smithi178 ceph-mon[33233]: from='osd.5 ' entity='osd.5' 2024-04-25T12:59:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:16 smithi038 ceph-mon[29400]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:16 smithi038 ceph-mon[29400]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T12:59:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:16 smithi038 ceph-mon[29400]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T12:59:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:17.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:17.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:16 smithi038 ceph-mon[29400]: from='osd.5 ' entity='osd.5' 2024-04-25T12:59:17.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:16 smithi104 ceph-mon[32726]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:59:17.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:16 smithi104 ceph-mon[32726]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T12:59:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:16 smithi104 ceph-mon[32726]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T12:59:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:17.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:16 smithi104 ceph-mon[32726]: from='osd.5 ' entity='osd.5' 2024-04-25T12:59:17.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-04-25T12:59:17.924 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] boot 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:18.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:17 smithi038 ceph-mon[29400]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] boot 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:17 smithi104 ceph-mon[32726]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:59:18.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: osd.5 [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] boot 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:18.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:17 smithi178 ceph-mon[33233]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:59:19.012 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:18 smithi178 ceph-mon[33233]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:19.012 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:18 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi178 2024-04-25T12:59:19.012 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:18 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi178 to 8544M 2024-04-25T12:59:19.012 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:18 smithi178 ceph-mon[33233]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:59:19.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:18 smithi038 ceph-mon[29400]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:19.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:18 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi178 2024-04-25T12:59:19.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:18 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi178 to 8544M 2024-04-25T12:59:19.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:18 smithi038 ceph-mon[29400]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:59:19.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:18 smithi104 ceph-mon[32726]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:19.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:18 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi178 2024-04-25T12:59:19.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:18 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi178 to 8544M 2024-04-25T12:59:19.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:18 smithi104 ceph-mon[32726]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 -- 172.21.15.38:0/2445014790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990103550 msgr2=0x7f2990105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 --2- 172.21.15.38:0/2445014790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990103550 0x7f2990105940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2980009990 tx=0x7f2980009ca0 comp rx=0 tx=0).stop 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 -- 172.21.15.38:0/2445014790 shutdown_connections 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 --2- 172.21.15.38:0/2445014790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 0x7f299010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 --2- 172.21.15.38:0/2445014790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990103550 0x7f2990105940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 --2- 172.21.15.38:0/2445014790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990105e80 0x7f2990108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 -- 172.21.15.38:0/2445014790 >> 172.21.15.38:0/2445014790 conn(0x7f29900fcf60 msgr2=0x7f29900ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 -- 172.21.15.38:0/2445014790 shutdown_connections 2024-04-25T12:59:19.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.724+0000 7f2994815700 1 -- 172.21.15.38:0/2445014790 wait complete. 2024-04-25T12:59:19.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 Processor -- start 2024-04-25T12:59:19.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 -- start start 2024-04-25T12:59:19.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990103550 0x7f2990102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:19.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 0x7f2990101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 0x7f2990101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f299010e680 con 0x7f2990105e80 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f299010e500 con 0x7f2990103550 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f2994815700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f299010e800 con 0x7f29901087b0 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 0x7f2990101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 0x7f2990101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36264/0 (socket says 172.21.15.38:36264) 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 -- 172.21.15.38:0/3465004139 learned_addr learned my addr 172.21.15.38:0/3465004139 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298ed9d700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 0x7f2990101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298e59c700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990103550 0x7f2990102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 -- 172.21.15.38:0/3465004139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 msgr2=0x7f2990101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 0x7f2990101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 -- 172.21.15.38:0/3465004139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990103550 msgr2=0x7f2990102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990103550 0x7f2990102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298dd9b700 1 -- 172.21.15.38:0/3465004139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29901022f0 con 0x7f2990105e80 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298e59c700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990103550 0x7f2990102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.725+0000 7f298ed9d700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 0x7f2990101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.726+0000 7f298dd9b700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 0x7f2990101500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f298400d980 tx=0x7f298400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.726+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984016070 con 0x7f2990105e80 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.726+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2984004d00 con 0x7f2990105e80 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.726+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29901a2e20 con 0x7f2990105e80 2024-04-25T12:59:19.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.726+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984005610 con 0x7f2990105e80 2024-04-25T12:59:19.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.726+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29901a3280 con 0x7f2990105e80 2024-04-25T12:59:19.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.727+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2990069730 con 0x7f2990105e80 2024-04-25T12:59:19.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.728+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2984004750 con 0x7f2990105e80 2024-04-25T12:59:19.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.729+0000 7f297f7fe700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f29780a3c40 0x7f29780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:19.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.729+0000 7f298e59c700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f29780a3c40 0x7f29780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:19.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.729+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f29840cf900 con 0x7f2990105e80 2024-04-25T12:59:19.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.729+0000 7f298e59c700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f29780a3c40 0x7f29780a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f298000b2e0 tx=0x7f298001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:19.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.733+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f298401f020 con 0x7f2990105e80 2024-04-25T12:59:19.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:19.952+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2990062e80 con 0x7f29780a3c40 2024-04-25T12:59:20.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:20 smithi038 ceph-mon[29400]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:20.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:20 smithi038 ceph-mon[29400]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:59:20.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:20 smithi038 ceph-mon[29400]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:20.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:20 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:20.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:20 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:20.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:20 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:21.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:20 smithi104 ceph-mon[32726]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:21.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:20 smithi104 ceph-mon[32726]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:59:21.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:20 smithi104 ceph-mon[32726]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:21.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:20 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:21.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:20 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:21.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:20 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:20 smithi178 ceph-mon[33233]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:20 smithi178 ceph-mon[33233]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:59:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:20 smithi178 ceph-mon[33233]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:20 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:20 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:21.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:20 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:22.052 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:21 smithi038 ceph-mon[29400]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T12:59:22.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:21 smithi104 ceph-mon[32726]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T12:59:22.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:21 smithi178 ceph-mon[33233]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T12:59:23.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:22 smithi038 ceph-mon[29400]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:23.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:22 smithi104 ceph-mon[32726]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:23.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:22 smithi178 ceph-mon[33233]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 213 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:24.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:23 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/786637161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcf8f123-b8f9-49e5-bca5-34f32285c0a5"}]: dispatch 2024-04-25T12:59:24.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:23 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/786637161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcf8f123-b8f9-49e5-bca5-34f32285c0a5"}]': finished 2024-04-25T12:59:24.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:23 smithi038 ceph-mon[29400]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:59:24.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:23 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:24.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:23 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3355763539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:24.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:23 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/786637161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcf8f123-b8f9-49e5-bca5-34f32285c0a5"}]: dispatch 2024-04-25T12:59:24.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:23 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/786637161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcf8f123-b8f9-49e5-bca5-34f32285c0a5"}]': finished 2024-04-25T12:59:24.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:23 smithi104 ceph-mon[32726]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:59:24.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:23 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:24.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:23 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3355763539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:24.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:23 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/786637161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcf8f123-b8f9-49e5-bca5-34f32285c0a5"}]: dispatch 2024-04-25T12:59:24.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:23 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/786637161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcf8f123-b8f9-49e5-bca5-34f32285c0a5"}]': finished 2024-04-25T12:59:24.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:23 smithi178 ceph-mon[33233]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:59:24.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:23 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:24.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:23 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3355763539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:25.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:25 smithi038 ceph-mon[29400]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T12:59:25.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:25 smithi104 ceph-mon[32726]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T12:59:25.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:25 smithi178 ceph-mon[33233]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T12:59:27.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:27 smithi038 ceph-mon[29400]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:59:27.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:27 smithi104 ceph-mon[32726]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:59:27.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:27 smithi178 ceph-mon[33233]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:59:29.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:29 smithi038 ceph-mon[29400]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T12:59:29.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:29 smithi104 ceph-mon[32726]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T12:59:29.449 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:29 smithi178 ceph-mon[33233]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T12:59:31.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:31 smithi038 ceph-mon[29400]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T12:59:31.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:59:31.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:31.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:31 smithi104 ceph-mon[32726]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T12:59:31.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:59:31.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:31.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:31 smithi178 ceph-mon[33233]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T12:59:31.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:59:31.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:32.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:32 smithi038 ceph-mon[29400]: Deploying daemon osd.6 on smithi038 2024-04-25T12:59:32.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:32 smithi104 ceph-mon[32726]: Deploying daemon osd.6 on smithi038 2024-04-25T12:59:32.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:32 smithi178 ceph-mon[33233]: Deploying daemon osd.6 on smithi038 2024-04-25T12:59:33.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:33 smithi038 ceph-mon[29400]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:59:33.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:33 smithi104 ceph-mon[32726]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:59:33.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:33 smithi178 ceph-mon[33233]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:59:35.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:35 smithi038 ceph-mon[29400]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:35.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:35 smithi104 ceph-mon[32726]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:35.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:35 smithi178 ceph-mon[33233]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:37.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:37 smithi038 ceph-mon[29400]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:37.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:37.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:37.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:37.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:37 smithi104 ceph-mon[32726]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:37.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:37.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:37.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:37.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:37 smithi178 ceph-mon[33233]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:37.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:37.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:39.065 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:39 smithi038 ceph-mon[29400]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:39.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:39 smithi104 ceph-mon[32726]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:39.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:39 smithi178 ceph-mon[33233]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:40.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.967+0000 7f297f7fe700 1 -- 172.21.15.38:0/3465004139 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2990062e80 con 0x7f29780a3c40 2024-04-25T12:59:40.965 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 6 on host 'smithi038' 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f29780a3c40 msgr2=0x7f29780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f29780a3c40 0x7f29780a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f298000b2e0 tx=0x7f298001a040 comp rx=0 tx=0).stop 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 msgr2=0x7f2990101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 0x7f2990101500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f298400d980 tx=0x7f298400dc90 comp rx=0 tx=0).stop 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 shutdown_connections 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f29780a3c40 0x7f29780a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2990103550 0x7f2990102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29901087b0 0x7f2990101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 --2- 172.21.15.38:0/3465004139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2990105e80 0x7f2990101500 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 >> 172.21.15.38:0/3465004139 conn(0x7f29900fcf60 msgr2=0x7f2990103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 shutdown_connections 2024-04-25T12:59:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T12:59:40.969+0000 7f2994815700 1 -- 172.21.15.38:0/3465004139 wait complete. 2024-04-25T12:59:41.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:41.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:41.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:59:41.778 DEBUG:teuthology.orchestra.run.smithi038:osd.6> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T12:59:41.781 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2024-04-25T12:59:41.781 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:59:42.026 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:59:42.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:42 smithi038 ceph-mon[29400]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:59:42.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:42 smithi038 ceph-mon[29400]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:59:42.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:42 smithi038 ceph-mon[29400]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:59:42.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:42.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:42 smithi104 ceph-mon[32726]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:59:42.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:42 smithi104 ceph-mon[32726]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:59:42.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:42 smithi104 ceph-mon[32726]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:59:42.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:42.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:42 smithi178 ceph-mon[33233]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:59:42.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:42 smithi178 ceph-mon[33233]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:59:42.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:42 smithi178 ceph-mon[33233]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T12:59:42.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:42.820 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 12:59:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T12:59:42.482+0000 7fd198ffa700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T12:59:42.821 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 12:59:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T12:59:42.491+0000 7fd190de5700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:59:43.247 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:43 smithi038 ceph-mon[29400]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:43.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:43 smithi038 ceph-mon[29400]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:59:43.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:43 smithi038 ceph-mon[29400]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T12:59:43.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:43.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:43.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:43 smithi038 ceph-mon[29400]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' 2024-04-25T12:59:43.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:43 smithi104 ceph-mon[32726]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:43.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:43 smithi104 ceph-mon[32726]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:59:43.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:43 smithi104 ceph-mon[32726]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T12:59:43.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:43.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:43.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:43 smithi104 ceph-mon[32726]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' 2024-04-25T12:59:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:43 smithi178 ceph-mon[33233]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:43 smithi178 ceph-mon[33233]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T12:59:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:43 smithi178 ceph-mon[33233]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T12:59:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:43 smithi178 ceph-mon[33233]: from='osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702]' entity='osd.6' 2024-04-25T12:59:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:59:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:59:44.209 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-25T12:59:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-25T12:59:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128278 s, 817 MB/s 2024-04-25T12:59:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-25T12:59:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:44 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T12:59:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:44 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T12:59:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:44 smithi178 ceph-mon[33233]: osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] boot 2024-04-25T12:59:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:44 smithi178 ceph-mon[33233]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:59:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:44.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:44 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T12:59:44.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:44 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T12:59:44.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:44.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:44 smithi104 ceph-mon[32726]: osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] boot 2024-04-25T12:59:44.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:44 smithi104 ceph-mon[32726]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:59:44.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:44 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T12:59:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:44 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T12:59:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:44 smithi038 ceph-mon[29400]: osd.6 [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] boot 2024-04-25T12:59:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:44 smithi038 ceph-mon[29400]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:59:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:59:44.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-04-25T12:59:45.202 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:59:45.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:45 smithi104 ceph-mon[32726]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:45.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:45 smithi104 ceph-mon[32726]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T12:59:45.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:45 smithi178 ceph-mon[33233]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:45.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:45 smithi178 ceph-mon[33233]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T12:59:45.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:45 smithi038 ceph-mon[29400]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:59:45.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:45 smithi038 ceph-mon[29400]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi038 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:46 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi038 2024-04-25T12:59:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:46.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:46.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:46 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi038 2024-04-25T12:59:46.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:59:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:59:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:46 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 -- 172.21.15.104:0/4105243361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 msgr2=0x7f7fbc108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 --2- 172.21.15.104:0/4105243361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc108250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009990 tx=0x7f7fa8009ca0 comp rx=0 tx=0).stop 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 -- 172.21.15.104:0/4105243361 shutdown_connections 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 --2- 172.21.15.104:0/4105243361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fbc108790 0x7f7fbc10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 --2- 172.21.15.104:0/4105243361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fbc103530 0x7f7fbc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 --2- 172.21.15.104:0/4105243361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc108250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 -- 172.21.15.104:0/4105243361 >> 172.21.15.104:0/4105243361 conn(0x7f7fbc0fcf40 msgr2=0x7f7fbc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 -- 172.21.15.104:0/4105243361 shutdown_connections 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.887+0000 7f7fc3238700 1 -- 172.21.15.104:0/4105243361 wait complete. 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 Processor -- start 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 -- start start 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fbc103530 0x7f7fbc198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fbc108790 0x7f7fbc1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc10e680 con 0x7f7fbc108790 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc10e500 con 0x7f7fbc105e60 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc3238700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc10e800 con 0x7f7fbc103530 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39742/0 (socket says 172.21.15.104:39742) 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 -- 172.21.15.104:0/2980151003 learned_addr learned my addr 172.21.15.104:0/2980151003 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc17d5700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fbc108790 0x7f7fbc1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 -- 172.21.15.104:0/2980151003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fbc103530 msgr2=0x7f7fbc198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fc0fd4700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fbc103530 0x7f7fbc198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fbc103530 0x7f7fbc198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 -- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fbc108790 msgr2=0x7f7fbc1a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fbc108790 0x7f7fbc1a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 -- 172.21.15.104:0/2980151003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc1a0960 con 0x7f7fbc105e60 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.888+0000 7f7fbbfff700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc199250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8006010 tx=0x7f7fa80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.889+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8027070 con 0x7f7fbc105e60 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.889+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7fa8004090 con 0x7f7fbc105e60 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.889+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8022e00 con 0x7f7fbc105e60 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.889+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc1a0bf0 con 0x7f7fbc105e60 2024-04-25T12:59:46.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.889+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1a1040 con 0x7f7fbc105e60 2024-04-25T12:59:46.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.890+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc069730 con 0x7f7fbc105e60 2024-04-25T12:59:46.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.891+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7fa8004240 con 0x7f7fbc105e60 2024-04-25T12:59:46.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.891+0000 7f7fb9ffb700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7fac0a3c40 0x7f7fac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:59:46.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.892+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f7fa80d08a0 con 0x7f7fbc105e60 2024-04-25T12:59:46.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.892+0000 7f7fc0fd4700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7fac0a3c40 0x7f7fac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:59:46.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.892+0000 7f7fc0fd4700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7fac0a3c40 0x7f7fac0a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005950 tx=0x7f7fb0005d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:59:46.893 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:46.895+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fa8030070 con 0x7f7fbc105e60 2024-04-25T12:59:47.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T12:59:47.105+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7fbc062e80 con 0x7f7fac0a3c40 2024-04-25T12:59:47.389 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:47 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi038 to 5697M 2024-04-25T12:59:47.389 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:47 smithi178 ceph-mon[33233]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:47.390 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:47.390 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:47.390 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:47.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:47 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi038 to 5697M 2024-04-25T12:59:47.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:47 smithi038 ceph-mon[29400]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:47.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:47.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:47.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:47.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:47 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi038 to 5697M 2024-04-25T12:59:47.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:47 smithi104 ceph-mon[32726]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:47.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:59:47.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:59:47.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:48.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:48 smithi104 ceph-mon[32726]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:48.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:48 smithi178 ceph-mon[33233]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:48.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:48 smithi038 ceph-mon[29400]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:59:49.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:49 smithi104 ceph-mon[32726]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:49.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:49 smithi178 ceph-mon[33233]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:49.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:49 smithi038 ceph-mon[29400]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:50.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:50 smithi178 ceph-mon[33233]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:50.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:50 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/3189091798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]: dispatch 2024-04-25T12:59:50.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:50 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]: dispatch 2024-04-25T12:59:50.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:50 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]': finished 2024-04-25T12:59:50.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:50 smithi178 ceph-mon[33233]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:59:50.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:59:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:50 smithi038 ceph-mon[29400]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:50 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/3189091798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]: dispatch 2024-04-25T12:59:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:50 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]: dispatch 2024-04-25T12:59:50.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:50 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]': finished 2024-04-25T12:59:50.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:50 smithi038 ceph-mon[29400]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:59:50.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:59:50.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:50 smithi104 ceph-mon[32726]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:50.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:50 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/3189091798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]: dispatch 2024-04-25T12:59:50.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:50 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]: dispatch 2024-04-25T12:59:50.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:50 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f074366-1e63-4ad6-9f21-30018b0cb493"}]': finished 2024-04-25T12:59:50.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:50 smithi104 ceph-mon[32726]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:59:50.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:59:51.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:51 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/2558456105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:51.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:51 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/2558456105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:51.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:51 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/2558456105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:59:52.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:52 smithi178 ceph-mon[33233]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:52.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:52 smithi038 ceph-mon[29400]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:52.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:52 smithi104 ceph-mon[32726]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:54.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:54 smithi038 ceph-mon[29400]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:54.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:54 smithi104 ceph-mon[32726]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:54.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:54 smithi178 ceph-mon[33233]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:56.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:56 smithi038 ceph-mon[29400]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:56.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:56 smithi104 ceph-mon[32726]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:56.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:56 smithi178 ceph-mon[33233]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:58 smithi178 ceph-mon[33233]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:59:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 12:59:58 smithi178 ceph-mon[33233]: Deploying daemon osd.7 on smithi104 2024-04-25T12:59:58.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:58 smithi038 ceph-mon[29400]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:59:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 12:59:58 smithi038 ceph-mon[29400]: Deploying daemon osd.7 on smithi104 2024-04-25T12:59:58.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:58 smithi104 ceph-mon[32726]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:59:58.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:59:58.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:59:58.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 12:59:58 smithi104 ceph-mon[32726]: Deploying daemon osd.7 on smithi104 2024-04-25T13:00:00.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:00 smithi038 ceph-mon[29400]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:00.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:00 smithi038 ceph-mon[29400]: overall HEALTH_OK 2024-04-25T13:00:00.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:00 smithi104 ceph-mon[32726]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:00.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:00 smithi104 ceph-mon[32726]: overall HEALTH_OK 2024-04-25T13:00:00.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:00 smithi178 ceph-mon[33233]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:00.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:00 smithi178 ceph-mon[33233]: overall HEALTH_OK 2024-04-25T13:00:02.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:02 smithi038 ceph-mon[29400]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:02.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:02 smithi104 ceph-mon[32726]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:02.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:02 smithi178 ceph-mon[33233]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:04.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:04 smithi104 ceph-mon[32726]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:04.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:04 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:04.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:04 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:04.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:04 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:04 smithi178 ceph-mon[33233]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:04 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:04 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:04 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:04.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:04 smithi038 ceph-mon[29400]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:04.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:04.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:04.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:06.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:06 smithi104 ceph-mon[32726]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:06.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:06 smithi178 ceph-mon[33233]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:06.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:06 smithi038 ceph-mon[29400]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:07.372 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.375+0000 7f7fb9ffb700 1 -- 172.21.15.104:0/2980151003 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7fbc062e80 con 0x7f7fac0a3c40 2024-04-25T13:00:07.372 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-04-25T13:00:07.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7fac0a3c40 msgr2=0x7f7fac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7fac0a3c40 0x7f7fac0a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005950 tx=0x7f7fb0005d10 comp rx=0 tx=0).stop 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 msgr2=0x7f7fbc199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc199250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8006010 tx=0x7f7fa80047b0 comp rx=0 tx=0).stop 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 shutdown_connections 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7fac0a3c40 0x7f7fac0a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fbc108790 0x7f7fbc1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fbc103530 0x7f7fbc198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 --2- 172.21.15.104:0/2980151003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fbc105e60 0x7f7fbc199250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 >> 172.21.15.104:0/2980151003 conn(0x7f7fbc0fcf40 msgr2=0x7f7fbc104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 shutdown_connections 2024-04-25T13:00:07.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:00:07.377+0000 7f7fc3238700 1 -- 172.21.15.104:0/2980151003 wait complete. 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:07.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:08.074 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T13:00:08.076 INFO:tasks.cephadm:Deploying osd.8 on smithi178 with /dev/vg_nvme/lv_2... 2024-04-25T13:00:08.077 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:00:08.326 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:00:08.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-mon[32726]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:08.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-mon[32726]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:00:08.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-mon[32726]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:00:08.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:08.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-mon[32726]: from='osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:00:08.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-mon[32726]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:00:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:08 smithi178 ceph-mon[33233]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:08 smithi178 ceph-mon[33233]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:00:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:08 smithi178 ceph-mon[33233]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:00:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:08 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:08 smithi178 ceph-mon[33233]: from='osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:00:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:08 smithi178 ceph-mon[33233]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:00:08.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:08 smithi038 ceph-mon[29400]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:08.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:08 smithi038 ceph-mon[29400]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:00:08.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:08 smithi038 ceph-mon[29400]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:00:08.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:08 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:08.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:08 smithi038 ceph-mon[29400]: from='osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:00:08.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:08 smithi038 ceph-mon[29400]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:00:09.323 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:00:08.897+0000 7f5bc199f700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:00:09.323 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:00:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:00:08.907+0000 7f5bb978a700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:00:09.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T13:00:09.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:00:09.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:09.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: from='osd.7 ' entity='osd.7' 2024-04-25T13:00:09.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] boot 2024-04-25T13:00:09.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T13:00:09.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:09 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: from='osd.7 ' entity='osd.7' 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] boot 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T13:00:09.705 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:09 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:09.708 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T13:00:09.708 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:00:09.708 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:09.709 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: from='osd.7 ' entity='osd.7' 2024-04-25T13:00:09.709 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: osd.7 [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] boot 2024-04-25T13:00:09.709 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T13:00:09.709 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:09 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:00:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:00:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:00:10.287 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-04-25T13:00:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-04-25T13:00:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0087114 s, 1.2 GB/s 2024-04-25T13:00:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-04-25T13:00:11.013 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-04-25T13:00:11.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:10 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T13:00:11.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:10 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T13:00:11.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:10 smithi104 ceph-mon[32726]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:11.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:10 smithi104 ceph-mon[32726]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:00:11.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:10 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T13:00:11.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:10 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T13:00:11.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:10 smithi178 ceph-mon[33233]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:11.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:10 smithi178 ceph-mon[33233]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:00:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:00:11.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:10 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T13:00:11.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:10 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T13:00:11.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:10 smithi038 ceph-mon[29400]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:00:11.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:10 smithi038 ceph-mon[29400]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:00:12.411 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi104 2024-04-25T13:00:12.411 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.411 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi104 to 5696M 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:12.412 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:12 smithi178 ceph-mon[33233]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:00:12.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi104 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi104 to 5696M 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:12.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:12 smithi038 ceph-mon[29400]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi104 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi104 to 5696M 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:12 smithi104 ceph-mon[32726]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.952+0000 7f59e9c6b700 1 -- 172.21.15.178:0/4279180195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4103c90 msgr2=0x7f59e4106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.952+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/4279180195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4103c90 0x7f59e4106120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b0a0 tx=0x7f59dc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 -- 172.21.15.178:0/4279180195 shutdown_connections 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/4279180195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59e4101320 0x7f59e4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/4279180195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 0x7f59e4106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/4279180195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4103c90 0x7f59e4106120 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 -- 172.21.15.178:0/4279180195 >> 172.21.15.178:0/4279180195 conn(0x7f59e40fcf40 msgr2=0x7f59e40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 -- 172.21.15.178:0/4279180195 shutdown_connections 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 -- 172.21.15.178:0/4279180195 wait complete. 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 Processor -- start 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 -- start start 2024-04-25T13:00:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 0x7f59e419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.953+0000 7f59e9c6b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59e4103c90 0x7f59e419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e9c6b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 0x7f59e41a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e9c6b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114bd0 con 0x7f59e4106660 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e9c6b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114a50 con 0x7f59e4103c90 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e9c6b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114d50 con 0x7f59e4101320 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 0x7f59e419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 0x7f59e419f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52536/0 (socket says 172.21.15.178:52536) 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 -- 172.21.15.178:0/3520800043 learned_addr learned my addr 172.21.15.178:0/3520800043 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e2ffd700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59e4103c90 0x7f59e419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e3fff700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 0x7f59e41a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 -- 172.21.15.178:0/3520800043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59e4103c90 msgr2=0x7f59e419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59e4103c90 0x7f59e419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 -- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 msgr2=0x7f59e41a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 0x7f59e41a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 -- 172.21.15.178:0/3520800043 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e41a6eb0 con 0x7f59e4101320 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e3fff700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 0x7f59e41a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e37fe700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 0x7f59e419f340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f59d400d9b0 tx=0x7f59d400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4016070 con 0x7f59e4101320 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e41a71a0 con 0x7f59e4101320 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59d4004d00 con 0x7f59e4101320 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4005650 con 0x7f59e4101320 2024-04-25T13:00:12.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.954+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e41a76e0 con 0x7f59e4101320 2024-04-25T13:00:12.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.955+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e4069730 con 0x7f59e4101320 2024-04-25T13:00:12.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.957+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59d4004750 con 0x7f59e4101320 2024-04-25T13:00:12.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.957+0000 7f59e0ff9700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f59cc0a3c40 0x7f59cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:12.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.957+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f59d40cfc10 con 0x7f59e4101320 2024-04-25T13:00:12.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.957+0000 7f59e2ffd700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f59cc0a3c40 0x7f59cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:12.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.958+0000 7f59e2ffd700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f59cc0a3c40 0x7f59cc0a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b830 tx=0x7f59dc006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:12.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:12.961+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59d401f020 con 0x7f59e4101320 2024-04-25T13:00:13.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:13.167+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f59e4062e80 con 0x7f59cc0a3c40 2024-04-25T13:00:13.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:13 smithi178 ceph-mon[33233]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:00:13.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:13.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:13.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:13.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:13 smithi038 ceph-mon[29400]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:00:13.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:13.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:13.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:13.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:13 smithi104 ceph-mon[32726]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:00:13.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:13.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:13.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:14 smithi178 ceph-mon[33233]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:14 smithi178 ceph-mon[33233]: pgmap v184: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:00:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:14 smithi178 ceph-mon[33233]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T13:00:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:14 smithi178 ceph-mon[33233]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:00:14.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:14 smithi038 ceph-mon[29400]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:14.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:14 smithi038 ceph-mon[29400]: pgmap v184: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:00:14.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:14 smithi038 ceph-mon[29400]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T13:00:14.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:14 smithi038 ceph-mon[29400]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:00:14.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:14 smithi104 ceph-mon[32726]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:14.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:14 smithi104 ceph-mon[32726]: pgmap v184: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:00:14.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:14 smithi104 ceph-mon[32726]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T13:00:14.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:14 smithi104 ceph-mon[32726]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:00:16.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:16 smithi178 ceph-mon[33233]: pgmap v186: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:00:16.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:16 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/1145178906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]: dispatch 2024-04-25T13:00:16.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:16 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]: dispatch 2024-04-25T13:00:16.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:16 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]': finished 2024-04-25T13:00:16.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:16 smithi178 ceph-mon[33233]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T13:00:16.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:16.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:16 smithi038 ceph-mon[29400]: pgmap v186: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:00:16.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:16 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/1145178906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]: dispatch 2024-04-25T13:00:16.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:16 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]: dispatch 2024-04-25T13:00:16.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:16 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]': finished 2024-04-25T13:00:16.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:16 smithi038 ceph-mon[29400]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T13:00:16.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:16.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:16 smithi104 ceph-mon[32726]: pgmap v186: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:00:16.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:16 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/1145178906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]: dispatch 2024-04-25T13:00:16.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:16 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]: dispatch 2024-04-25T13:00:16.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:16 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d79a36-ce86-4e9a-b60a-6a709d437bc4"}]': finished 2024-04-25T13:00:16.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:16 smithi104 ceph-mon[32726]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T13:00:16.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:17.724 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:17 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/2353632170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:17.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:17 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/2353632170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:17.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:17 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/2353632170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:18.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:18 smithi178 ceph-mon[33233]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:18.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:18 smithi178 ceph-mon[33233]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T13:00:18.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:18 smithi178 ceph-mon[33233]: Cluster is now healthy 2024-04-25T13:00:18.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:18 smithi038 ceph-mon[29400]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:18.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:18 smithi038 ceph-mon[29400]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T13:00:18.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:18 smithi038 ceph-mon[29400]: Cluster is now healthy 2024-04-25T13:00:18.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:18 smithi104 ceph-mon[32726]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:18.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:18 smithi104 ceph-mon[32726]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T13:00:18.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:18 smithi104 ceph-mon[32726]: Cluster is now healthy 2024-04-25T13:00:20.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:20 smithi178 ceph-mon[33233]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:20.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:20 smithi038 ceph-mon[29400]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:20.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:20 smithi104 ceph-mon[32726]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:22.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:22 smithi178 ceph-mon[33233]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:22.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:22 smithi038 ceph-mon[29400]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:22.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:22 smithi104 ceph-mon[32726]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:24.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:24 smithi178 ceph-mon[33233]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:24.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:00:24.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:24.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:24 smithi178 ceph-mon[33233]: Deploying daemon osd.8 on smithi178 2024-04-25T13:00:24.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:24 smithi038 ceph-mon[29400]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:00:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:24 smithi038 ceph-mon[29400]: Deploying daemon osd.8 on smithi178 2024-04-25T13:00:24.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:24 smithi104 ceph-mon[32726]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:00:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:24 smithi104 ceph-mon[32726]: Deploying daemon osd.8 on smithi178 2024-04-25T13:00:26.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:26 smithi178 ceph-mon[33233]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:26.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:26 smithi038 ceph-mon[29400]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:26.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:26 smithi104 ceph-mon[32726]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:28.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:28 smithi178 ceph-mon[33233]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:28.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:28 smithi038 ceph-mon[29400]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:28.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:28 smithi104 ceph-mon[32726]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:30.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:30 smithi178 ceph-mon[33233]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:30.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:30.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:30 smithi038 ceph-mon[29400]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:30.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:30.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:30.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:30.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:30 smithi104 ceph-mon[32726]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:30.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:30.822 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:30.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:32.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:32 smithi178 ceph-mon[33233]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:32.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:32.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:32.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:32 smithi038 ceph-mon[29400]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:32.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:32 smithi104 ceph-mon[32726]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:33.191 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 8 on host 'smithi178' 2024-04-25T13:00:33.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.191+0000 7f59e0ff9700 1 -- 172.21.15.178:0/3520800043 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f59e4062e80 con 0x7f59cc0a3c40 2024-04-25T13:00:33.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f59cc0a3c40 msgr2=0x7f59cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f59cc0a3c40 0x7f59cc0a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b830 tx=0x7f59dc006000 comp rx=0 tx=0).stop 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 msgr2=0x7f59e419f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 0x7f59e419f340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f59d400d9b0 tx=0x7f59d400dcc0 comp rx=0 tx=0).stop 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 shutdown_connections 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f59cc0a3c40 0x7f59cc0a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e4106660 0x7f59e41a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e4101320 0x7f59e419f340 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 --2- 172.21.15.178:0/3520800043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59e4103c90 0x7f59e419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 >> 172.21.15.178:0/3520800043 conn(0x7f59e40fcf40 msgr2=0x7f59e4102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 shutdown_connections 2024-04-25T13:00:33.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:00:33.194+0000 7f59e9c6b700 1 -- 172.21.15.178:0/3520800043 wait complete. 2024-04-25T13:00:33.849 DEBUG:teuthology.orchestra.run.smithi178:osd.8> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T13:00:33.851 INFO:tasks.cephadm:Deploying osd.9 on smithi038 with /dev/vg_nvme/lv_1... 2024-04-25T13:00:33.851 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:00:34.108 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:00:34.479 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:34 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:34.479 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:34 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:34.479 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:34 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:34.479 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:34 smithi038 ceph-mon[29400]: from='osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:00:34.479 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:34 smithi038 ceph-mon[29400]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:00:34.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:34 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:34.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:34 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:34.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:34 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:34.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:34 smithi178 ceph-mon[33233]: from='osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:00:34.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:34 smithi178 ceph-mon[33233]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:00:34.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:34 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:34.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:34 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:34.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:34 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:34.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:34 smithi104 ceph-mon[32726]: from='osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:00:34.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:34 smithi104 ceph-mon[32726]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:00:35.279 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-mon[33233]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:35.279 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-mon[33233]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:00:35.279 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-mon[33233]: from='osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:00:35.279 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-mon[33233]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:00:35.279 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:35.279 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-mon[33233]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:00:35.484 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:35 smithi038 ceph-mon[29400]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:35.484 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:35 smithi038 ceph-mon[29400]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:00:35.484 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:35 smithi038 ceph-mon[29400]: from='osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:00:35.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:35 smithi038 ceph-mon[29400]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:00:35.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:35 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:35.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:35 smithi038 ceph-mon[29400]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:00:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:35 smithi104 ceph-mon[32726]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:35 smithi104 ceph-mon[32726]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:00:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:35 smithi104 ceph-mon[32726]: from='osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:00:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:35 smithi104 ceph-mon[32726]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:00:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:35 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:35 smithi104 ceph-mon[32726]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:00:35.988 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:00:35.604+0000 7fa8d8750700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T13:00:35.989 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:00:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:00:35.615+0000 7fa8d053b700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: from='osd.8 ' entity='osd.8' 2024-04-25T13:00:36.351 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] boot 2024-04-25T13:00:36.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T13:00:36.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.379 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:00:36.379 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:00:36.379 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-25T13:00:36.379 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-25T13:00:36.379 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0162766 s, 644 MB/s 2024-04-25T13:00:36.379 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-25T13:00:36.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T13:00:36.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:00:36.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: from='osd.8 ' entity='osd.8' 2024-04-25T13:00:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] boot 2024-04-25T13:00:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T13:00:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: from='osd.8 ' entity='osd.8' 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: osd.8 [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] boot 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T13:00:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:00:37.168 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-04-25T13:00:37.422 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:00:37.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T13:00:37.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T13:00:37.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:00:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi178 2024-04-25T13:00:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi178 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi178 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:00:38.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi178 to 5696M 2024-04-25T13:00:38.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:38.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:38 smithi178 ceph-mon[33233]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:00:38.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi178 to 5696M 2024-04-25T13:00:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:38 smithi038 ceph-mon[29400]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi178 to 5696M 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:38 smithi104 ceph-mon[32726]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 -- 172.21.15.38:0/3914560496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 msgr2=0x7fc088074e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/3914560496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc088074e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc078009990 tx=0x7fc078009ca0 comp rx=0 tx=0).stop 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 -- 172.21.15.38:0/3914560496 shutdown_connections 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/3914560496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08810a980 0x7fc08810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/3914560496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc088074e70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/3914560496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc088075440 0x7fc0880758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 -- 172.21.15.38:0/3914560496 >> 172.21.15.38:0/3914560496 conn(0x7fc0880fe040 msgr2=0x7fc088100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 -- 172.21.15.38:0/3914560496 shutdown_connections 2024-04-25T13:00:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.237+0000 7fc08d4a6700 1 -- 172.21.15.38:0/3914560496 wait complete. 2024-04-25T13:00:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 Processor -- start 2024-04-25T13:00:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 -- start start 2024-04-25T13:00:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc088075440 0x7fc08819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc08819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08810a980 0x7fc0881a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc088110850 con 0x7fc088076a10 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881106d0 con 0x7fc08810a980 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc08d4a6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881109d0 con 0x7fc088075440 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc0867fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc08819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc0867fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc08819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39270/0 (socket says 172.21.15.38:39270) 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.238+0000 7fc0867fc700 1 -- 172.21.15.38:0/1690470140 learned_addr learned my addr 172.21.15.38:0/1690470140 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0877fe700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08810a980 0x7fc0881a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc086ffd700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc088075440 0x7fc08819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0867fc700 1 -- 172.21.15.38:0/1690470140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc088075440 msgr2=0x7fc08819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0867fc700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc088075440 0x7fc08819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0867fc700 1 -- 172.21.15.38:0/1690470140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08810a980 msgr2=0x7fc0881a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0867fc700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08810a980 0x7fc0881a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:00:39.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0867fc700 1 -- 172.21.15.38:0/1690470140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0881a2ac0 con 0x7fc088076a10 2024-04-25T13:00:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc0867fc700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc08819b440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c9a0 tx=0x7fc07c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004a30 con 0x7fc088076a10 2024-04-25T13:00:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc07c0054f0 con 0x7fc088076a10 2024-04-25T13:00:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0881a2db0 con 0x7fc088076a10 2024-04-25T13:00:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c007cd0 con 0x7fc088076a10 2024-04-25T13:00:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.239+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881a3210 con 0x7fc088076a10 2024-04-25T13:00:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.240+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08802df80 con 0x7fc088076a10 2024-04-25T13:00:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.242+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc07c004cf0 con 0x7fc088076a10 2024-04-25T13:00:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.242+0000 7fc06ffff700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc0700a3c40 0x7fc0700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:00:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.242+0000 7fc086ffd700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc0700a3c40 0x7fc0700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:00:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.242+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fc07c0cfac0 con 0x7fc088076a10 2024-04-25T13:00:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.243+0000 7fc086ffd700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc0700a3c40 0x7fc0700a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b5c0 tx=0x7fc078005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:00:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.246+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc07c09cee0 con 0x7fc088076a10 2024-04-25T13:00:39.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:00:39.463+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc088062e80 con 0x7fc0700a3c40 2024-04-25T13:00:40.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:40 smithi038 ceph-mon[29400]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:00:40.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:40 smithi038 ceph-mon[29400]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:40.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:40 smithi038 ceph-mon[29400]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:40.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:40.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:40.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:40.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:40 smithi178 ceph-mon[33233]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:00:40.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:40 smithi178 ceph-mon[33233]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:40.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:40 smithi178 ceph-mon[33233]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:40.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:40.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:40.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:40 smithi104 ceph-mon[32726]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:00:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:40 smithi104 ceph-mon[32726]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:40 smithi104 ceph-mon[32726]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:00:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:00:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:00:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:41.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:41 smithi038 ceph-mon[29400]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:00:41.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:41 smithi178 ceph-mon[33233]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:00:41.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:41 smithi104 ceph-mon[32726]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:00:42.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:42 smithi178 ceph-mon[33233]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:00:42.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:42 smithi038 ceph-mon[29400]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:00:42.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:42 smithi104 ceph-mon[32726]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:00:43.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:43 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2627046856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47dbcc3a-1a6a-4896-a5d2-9866e080fd7b"}]: dispatch 2024-04-25T13:00:43.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:43 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2627046856' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47dbcc3a-1a6a-4896-a5d2-9866e080fd7b"}]': finished 2024-04-25T13:00:43.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:43 smithi178 ceph-mon[33233]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T13:00:43.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:00:43.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:43 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2627046856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47dbcc3a-1a6a-4896-a5d2-9866e080fd7b"}]: dispatch 2024-04-25T13:00:43.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:43 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2627046856' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47dbcc3a-1a6a-4896-a5d2-9866e080fd7b"}]': finished 2024-04-25T13:00:43.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:43 smithi038 ceph-mon[29400]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T13:00:43.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:00:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:43 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2627046856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47dbcc3a-1a6a-4896-a5d2-9866e080fd7b"}]: dispatch 2024-04-25T13:00:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:43 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2627046856' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47dbcc3a-1a6a-4896-a5d2-9866e080fd7b"}]': finished 2024-04-25T13:00:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:43 smithi104 ceph-mon[32726]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T13:00:43.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:00:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:44 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/908691302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:44.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:44 smithi178 ceph-mon[33233]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:00:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:44 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/908691302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:44.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:44 smithi038 ceph-mon[29400]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:00:44.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:44 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/908691302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:00:44.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:44 smithi104 ceph-mon[32726]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:00:46.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:46 smithi038 ceph-mon[29400]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:00:46.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:46 smithi104 ceph-mon[32726]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:00:46.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:46 smithi178 ceph-mon[33233]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:00:48.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:48 smithi038 ceph-mon[29400]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:00:48.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:48 smithi104 ceph-mon[32726]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:00:48.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:48 smithi178 ceph-mon[33233]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:00:50.495 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:50 smithi038 ceph-mon[29400]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:00:50.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:50 smithi104 ceph-mon[32726]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:00:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:50 smithi178 ceph-mon[33233]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:00:51.544 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:00:51.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:51 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:51.545 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:51 smithi038 ceph-mon[29400]: Deploying daemon osd.9 on smithi038 2024-04-25T13:00:51.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:51 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:00:51.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:51 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:51.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:51 smithi178 ceph-mon[33233]: Deploying daemon osd.9 on smithi038 2024-04-25T13:00:51.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:51 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:00:51.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:51 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:00:51.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:51 smithi104 ceph-mon[32726]: Deploying daemon osd.9 on smithi038 2024-04-25T13:00:52.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:52 smithi038 ceph-mon[29400]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:52.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:52 smithi178 ceph-mon[33233]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:52.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:52 smithi104 ceph-mon[32726]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:54.537 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:54 smithi038 ceph-mon[29400]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:54.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:54 smithi178 ceph-mon[33233]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:54.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:54 smithi104 ceph-mon[32726]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:56.486 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:56 smithi038 ceph-mon[29400]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:56.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:56 smithi178 ceph-mon[33233]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:56.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:56 smithi104 ceph-mon[32726]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:58.287 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:58.287 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:58.288 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:58 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:58 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:00:58.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:58 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:00:59.179 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:00:59 smithi038 ceph-mon[29400]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:00:59 smithi104 ceph-mon[32726]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:00:59.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:00:59 smithi178 ceph-mon[33233]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:01.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:01.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:01.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:01.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.257 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:01 smithi038 ceph-mon[29400]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:01 smithi104 ceph-mon[32726]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:01:01.325 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 9 on host 'smithi038' 2024-04-25T13:01:01.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.327+0000 7fc06ffff700 1 -- 172.21.15.38:0/1690470140 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc088062e80 con 0x7fc0700a3c40 2024-04-25T13:01:01.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.329+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc0700a3c40 msgr2=0x7fc0700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:01.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.329+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc0700a3c40 0x7fc0700a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b5c0 tx=0x7fc078005fb0 comp rx=0 tx=0).stop 2024-04-25T13:01:01.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.329+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 msgr2=0x7fc08819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:01.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.329+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc08819b440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c9a0 tx=0x7fc07c00ccb0 comp rx=0 tx=0).stop 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 shutdown_connections 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc0700a3c40 0x7fc0700a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08810a980 0x7fc0881a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc088075440 0x7fc08819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 --2- 172.21.15.38:0/1690470140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc088076a10 0x7fc08819b440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 >> 172.21.15.38:0/1690470140 conn(0x7fc0880fe040 msgr2=0x7fc0881068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 shutdown_connections 2024-04-25T13:01:01.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:01.330+0000 7fc08d4a6700 1 -- 172.21.15.38:0/1690470140 wait complete. 2024-04-25T13:01:01.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:01.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:01.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:01.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:01.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:01 smithi178 ceph-mon[33233]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:01:02.118 DEBUG:teuthology.orchestra.run.smithi038:osd.9> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T13:01:02.120 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2024-04-25T13:01:02.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:01:02.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:02.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:02.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:02.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:02.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:01:02.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:01:02.369 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T13:01:02.369 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:02.369 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:01:02.490 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:02.490 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:02.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:02.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:02.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:01:02.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:01:02.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T13:01:02.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:02 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:02.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:02.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:02.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:01:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:01:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-25T13:01:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:02 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.320 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:01:02.944+0000 7f4470970700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T13:01:03.321 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:01:02 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:01:02.957+0000 7f446875b700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:01:03.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:03 smithi178 ceph-mon[33233]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T13:01:03.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:03 smithi178 ceph-mon[33233]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:01:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:03 smithi178 ceph-mon[33233]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' 2024-04-25T13:01:03.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:03 smithi038 ceph-mon[29400]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T13:01:03.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:03 smithi038 ceph-mon[29400]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:01:03.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:03 smithi038 ceph-mon[29400]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' 2024-04-25T13:01:03.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:03 smithi104 ceph-mon[32726]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-25T13:01:03.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:03 smithi104 ceph-mon[32726]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:01:03.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:03.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:03 smithi104 ceph-mon[32726]: from='osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004]' entity='osd.9' 2024-04-25T13:01:04.580 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:01:04.580 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:01:04.580 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-25T13:01:04.580 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-25T13:01:04.580 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119067 s, 881 MB/s 2024-04-25T13:01:04.580 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-25T13:01:04.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T13:01:04.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T13:01:04.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:04.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:04.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] boot 2024-04-25T13:01:04.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T13:01:04.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:04 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T13:01:04.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T13:01:04.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:04.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:04.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] boot 2024-04-25T13:01:04.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T13:01:04.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:04 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: osd.9 [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] boot 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T13:01:04.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:04 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:01:05.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-04-25T13:01:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi038 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi038 to 4273M 2024-04-25T13:01:06.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:06.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:06.607 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:06.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:01:06.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi038 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi038 to 4273M 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:06.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:01:06.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi038 2024-04-25T13:01:06.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi038 to 4273M 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:06.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 -- 172.21.15.104:0/3212367185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210108850 msgr2=0x7f721010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 --2- 172.21.15.104:0/3212367185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210108850 0x7f721010ac40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f720800b0a0 tx=0x7f720800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 -- 172.21.15.104:0/3212367185 shutdown_connections 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 --2- 172.21.15.104:0/3212367185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210108850 0x7f721010ac40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 --2- 172.21.15.104:0/3212367185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72101035f0 0x7f72101059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 --2- 172.21.15.104:0/3212367185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7210105f20 0x7f7210108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 -- 172.21.15.104:0/3212367185 >> 172.21.15.104:0/3212367185 conn(0x7f72100fcf40 msgr2=0x7f72100ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.167+0000 7f7217252700 1 -- 172.21.15.104:0/3212367185 shutdown_connections 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 -- 172.21.15.104:0/3212367185 wait complete. 2024-04-25T13:01:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 Processor -- start 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 -- start start 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72101035f0 0x7f7210198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 0x7f72101991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7210108850 0x7f72101a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f721010e650 con 0x7f7210108850 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f721010e4d0 con 0x7f7210105f20 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f7217252700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f721010e7d0 con 0x7f72101035f0 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f720ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 0x7f72101991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f720ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 0x7f72101991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57758/0 (socket says 172.21.15.104:57758) 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.168+0000 7f720ffff700 1 -- 172.21.15.104:0/3738751328 learned_addr learned my addr 172.21.15.104:0/3738751328 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f72157ef700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7210108850 0x7f72101a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f7214fee700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72101035f0 0x7f7210198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720ffff700 1 -- 172.21.15.104:0/3738751328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72101035f0 msgr2=0x7f7210198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720ffff700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72101035f0 0x7f7210198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720ffff700 1 -- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7210108850 msgr2=0x7f72101a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720ffff700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7210108850 0x7f72101a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720ffff700 1 -- 172.21.15.104:0/3738751328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72101a0910 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f7214fee700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72101035f0 0x7f7210198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720ffff700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 0x7f72101991d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00c970 tx=0x7f71fc00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc007aa0 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72101a0c00 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f71fc004d00 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0055c0 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.169+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72101a10f0 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.170+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7210069730 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.171+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f71fc007c40 con 0x7f7210105f20 2024-04-25T13:01:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.172+0000 7f720dffb700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f72000a3c40 0x7f72000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.172+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f71fc0cf9c0 con 0x7f7210105f20 2024-04-25T13:01:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.172+0000 7f7214fee700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f72000a3c40 0x7f72000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.172+0000 7f7214fee700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f72000a3c40 0x7f72000a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7204005950 tx=0x7f72040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.176+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71fc0d1e20 con 0x7f7210105f20 2024-04-25T13:01:07.382 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:07.384+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7210062e80 con 0x7f72000a3c40 2024-04-25T13:01:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:01:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:01:07.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:07 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:01:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:01:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:07 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:01:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:01:07.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:07 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:08.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:08 smithi104 ceph-mon[32726]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:08.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:08 smithi104 ceph-mon[32726]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:08.738 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:08 smithi038 ceph-mon[29400]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:08.738 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:08 smithi038 ceph-mon[29400]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:08.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:08 smithi178 ceph-mon[33233]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:08.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:08 smithi178 ceph-mon[33233]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:10.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:10 smithi178 ceph-mon[33233]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:10 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/486548344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]: dispatch 2024-04-25T13:01:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:10 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]: dispatch 2024-04-25T13:01:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:10 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]': finished 2024-04-25T13:01:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:10 smithi178 ceph-mon[33233]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T13:01:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:10.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:10 smithi038 ceph-mon[29400]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:10.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:10 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/486548344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]: dispatch 2024-04-25T13:01:10.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:10 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]: dispatch 2024-04-25T13:01:10.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:10 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]': finished 2024-04-25T13:01:10.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:10 smithi038 ceph-mon[29400]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T13:01:10.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:10 smithi104 ceph-mon[32726]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:10 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/486548344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]: dispatch 2024-04-25T13:01:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:10 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]: dispatch 2024-04-25T13:01:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:10 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38c90875-61d4-402f-bee4-af2adb22e185"}]': finished 2024-04-25T13:01:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:10 smithi104 ceph-mon[32726]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T13:01:10.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:11.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:11 smithi178 ceph-mon[33233]: from='client.? 172.21.15.104:0/1277067982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:01:11.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:11 smithi038 ceph-mon[29400]: from='client.? 172.21.15.104:0/1277067982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:01:11.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:11 smithi104 ceph-mon[32726]: from='client.? 172.21.15.104:0/1277067982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:01:12.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:12 smithi178 ceph-mon[33233]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:12.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:12 smithi038 ceph-mon[29400]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:12.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:12 smithi104 ceph-mon[32726]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:14.560 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:14 smithi104 ceph-mon[32726]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:14 smithi178 ceph-mon[33233]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:14.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:14 smithi038 ceph-mon[29400]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:16 smithi104 ceph-mon[32726]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:16.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:16 smithi038 ceph-mon[29400]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:16.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:16 smithi178 ceph-mon[33233]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:18 smithi104 ceph-mon[32726]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:01:18.604 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:18.605 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:18 smithi104 ceph-mon[32726]: Deploying daemon osd.10 on smithi104 2024-04-25T13:01:18.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:18 smithi178 ceph-mon[33233]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:18.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:01:18.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:18.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:18 smithi178 ceph-mon[33233]: Deploying daemon osd.10 on smithi104 2024-04-25T13:01:18.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:18 smithi038 ceph-mon[29400]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:18.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:01:18.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:18.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:18 smithi038 ceph-mon[29400]: Deploying daemon osd.10 on smithi104 2024-04-25T13:01:20.774 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:20 smithi104 ceph-mon[32726]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:20.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:20 smithi038 ceph-mon[29400]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:20.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:20 smithi178 ceph-mon[33233]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:22.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:22 smithi038 ceph-mon[29400]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:22.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:22 smithi104 ceph-mon[32726]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:22.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:22 smithi178 ceph-mon[33233]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:24.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:24 smithi038 ceph-mon[29400]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:24 smithi104 ceph-mon[32726]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:24.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:24 smithi178 ceph-mon[33233]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:26.804 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:26 smithi104 ceph-mon[32726]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:26.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:26 smithi038 ceph-mon[29400]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:26.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:26 smithi178 ceph-mon[33233]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:27.656 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2024-04-25T13:01:27.656 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.658+0000 7f720dffb700 1 -- 172.21.15.104:0/3738751328 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7210062e80 con 0x7f72000a3c40 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.660+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f72000a3c40 msgr2=0x7f72000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.660+0000 7f7217252700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f72000a3c40 0x7f72000a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7204005950 tx=0x7f72040058e0 comp rx=0 tx=0).stop 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.660+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 msgr2=0x7f72101991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.660+0000 7f7217252700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 0x7f72101991d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00c970 tx=0x7f71fc00cc80 comp rx=0 tx=0).stop 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 shutdown_connections 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f72000a3c40 0x7f72000a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:27.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7210108850 0x7f72101a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:27.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72101035f0 0x7f7210198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:27.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 --2- 172.21.15.104:0/3738751328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7210105f20 0x7f72101991d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:27.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 >> 172.21.15.104:0/3738751328 conn(0x7f72100fcf40 msgr2=0x7f7210103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:27.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 shutdown_connections 2024-04-25T13:01:27.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-25T13:01:27.661+0000 7f7217252700 1 -- 172.21.15.104:0/3738751328 wait complete. 2024-04-25T13:01:27.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:01:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:27 smithi038 ceph-mon[29400]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:01:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:27 smithi104 ceph-mon[32726]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:01:27.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:01:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:27 smithi178 ceph-mon[33233]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:01:28.373 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T13:01:28.375 INFO:tasks.cephadm:Deploying osd.11 on smithi178 with /dev/vg_nvme/lv_1... 2024-04-25T13:01:28.375 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:01:28.598 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:01:28.593+0000 7f169abf1700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T13:01:28.598 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:28.598 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:28.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:28.624 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:01:28.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:28.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:28 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:28.925 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:01:28 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:01:28.608+0000 7f16969e4700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:29.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:28 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:29.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:29 smithi178 ceph-mon[33233]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T13:01:29.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:29 smithi178 ceph-mon[33233]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:01:29.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:29.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:29.916 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:29 smithi104 ceph-mon[32726]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T13:01:29.916 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:29 smithi104 ceph-mon[32726]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:01:29.917 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:29.917 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:29.980 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:29 smithi038 ceph-mon[29400]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-25T13:01:29.981 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:29 smithi038 ceph-mon[29400]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:01:29.981 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:29.981 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:30.651 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:01:30.651 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:01:30.651 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-04-25T13:01:30.651 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-04-25T13:01:30.651 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00861999 s, 1.2 GB/s 2024-04-25T13:01:30.651 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-04-25T13:01:30.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:30 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T13:01:30.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:30 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T13:01:30.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:30 smithi104 ceph-mon[32726]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:30.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:30 smithi104 ceph-mon[32726]: osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] boot 2024-04-25T13:01:30.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:30 smithi104 ceph-mon[32726]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T13:01:30.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:30.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:30 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T13:01:30.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:30 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T13:01:30.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:30 smithi178 ceph-mon[33233]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:30.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:30 smithi178 ceph-mon[33233]: osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] boot 2024-04-25T13:01:30.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:30 smithi178 ceph-mon[33233]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T13:01:30.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:31.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:30 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T13:01:31.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:30 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T13:01:31.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:30 smithi038 ceph-mon[29400]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:01:31.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:30 smithi038 ceph-mon[29400]: osd.10 [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] boot 2024-04-25T13:01:31.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:30 smithi038 ceph-mon[29400]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T13:01:31.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:01:31.346 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-04-25T13:01:31.589 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:01:31.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T13:01:31.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:31 smithi104 ceph-mon[32726]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.888 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:31.889 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.889 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:31.889 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:31.889 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:31.889 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:31 smithi178 ceph-mon[33233]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:01:32.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T13:01:32.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:31 smithi038 ceph-mon[29400]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:01:32.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:32 smithi178 ceph-mon[33233]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:32.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:32 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi104 2024-04-25T13:01:32.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:32 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi104 to 4272M 2024-04-25T13:01:33.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:32 smithi038 ceph-mon[29400]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:32 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi104 2024-04-25T13:01:33.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:32 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi104 to 4272M 2024-04-25T13:01:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:32 smithi104 ceph-mon[32726]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:32 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi104 2024-04-25T13:01:33.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:32 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi104 to 4272M 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3880887597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4101c60 msgr2=0x7fb4d41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3880887597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4101c60 0x7fb4d41020c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009990 tx=0x7fb4c0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3880887597 shutdown_connections 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3880887597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4102600 0x7fb4d4109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3880887597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 0x7fb4d41033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3880887597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4101c60 0x7fb4d41020c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3880887597 >> 172.21.15.178:0/3880887597 conn(0x7fb4d40fc6f0 msgr2=0x7fb4d40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3880887597 shutdown_connections 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.226+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3880887597 wait complete. 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 Processor -- start 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 -- start start 2024-04-25T13:01:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4101c60 0x7fb4d4197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 0x7fb4d4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 0x7fb4d419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410d690 con 0x7fb4d4102fb0 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410d510 con 0x7fb4d4101c60 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4dadc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410d810 con 0x7fb4d4102600 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 0x7fb4d4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 0x7fb4d4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59790/0 (socket says 172.21.15.178:59790) 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.227+0000 7fb4d3fff700 1 -- 172.21.15.178:0/3961368630 learned_addr learned my addr 172.21.15.178:0/3961368630 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d8b65700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4101c60 0x7fb4d4197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d9366700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 0x7fb4d419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d3fff700 1 -- 172.21.15.178:0/3961368630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4101c60 msgr2=0x7fb4d4197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d3fff700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4101c60 0x7fb4d4197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d3fff700 1 -- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 msgr2=0x7fb4d419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d3fff700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 0x7fb4d419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d3fff700 1 -- 172.21.15.178:0/3961368630 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d419f940 con 0x7fb4d4102600 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d9366700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 0x7fb4d419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d8b65700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4101c60 0x7fb4d4197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d3fff700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 0x7fb4d4198230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0006010 tx=0x7fb4c0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0027070 con 0x7fb4d4102600 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d419fbd0 con 0x7fb4d4102600 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb4c0022470 con 0x7fb4d4102600 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000f6e0 con 0x7fb4d4102600 2024-04-25T13:01:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.228+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a0080 con 0x7fb4d4102600 2024-04-25T13:01:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.229+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d4069730 con 0x7fb4d4102600 2024-04-25T13:01:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.230+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4c002a070 con 0x7fb4d4102600 2024-04-25T13:01:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.231+0000 7fb4d1ffb700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb4c40a3cc0 0x7fb4c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.231+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fb4c00d0f40 con 0x7fb4d4102600 2024-04-25T13:01:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.231+0000 7fb4d8b65700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb4c40a3cc0 0x7fb4c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:33.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.232+0000 7fb4d8b65700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb4c40a3cc0 0x7fb4c40a6180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4d41991d0 tx=0x7fb4c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:33.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.235+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4c0013070 con 0x7fb4d4102600 2024-04-25T13:01:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:33.443+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb4d4062e80 con 0x7fb4c40a3cc0 2024-04-25T13:01:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:33 smithi178 ceph-mon[33233]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:01:33.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:01:33.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:01:33.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:33 smithi038 ceph-mon[29400]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:01:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:01:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:01:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:34.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:33 smithi104 ceph-mon[32726]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:01:34.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:01:34.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:01:34.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:34.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:34 smithi178 ceph-mon[33233]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:34.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:34 smithi178 ceph-mon[33233]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:34.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:34 smithi178 ceph-mon[33233]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:01:35.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:34 smithi038 ceph-mon[29400]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:34 smithi038 ceph-mon[29400]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:35.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:34 smithi038 ceph-mon[29400]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:01:35.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:34 smithi104 ceph-mon[32726]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:35.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:34 smithi104 ceph-mon[32726]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:01:35.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:34 smithi104 ceph-mon[32726]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:01:36.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:36 smithi178 ceph-mon[33233]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:36.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:36 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/33412298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]: dispatch 2024-04-25T13:01:36.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:36 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]: dispatch 2024-04-25T13:01:36.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:36 smithi178 ceph-mon[33233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]': finished 2024-04-25T13:01:36.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:36 smithi178 ceph-mon[33233]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T13:01:36.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:37.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:36 smithi038 ceph-mon[29400]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:36 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/33412298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]: dispatch 2024-04-25T13:01:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:36 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]: dispatch 2024-04-25T13:01:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:36 smithi038 ceph-mon[29400]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]': finished 2024-04-25T13:01:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:36 smithi038 ceph-mon[29400]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T13:01:37.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:36 smithi104 ceph-mon[32726]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:36 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/33412298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]: dispatch 2024-04-25T13:01:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:36 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]: dispatch 2024-04-25T13:01:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:36 smithi104 ceph-mon[32726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f03efb23-83a5-460c-90be-6aef89f88698"}]': finished 2024-04-25T13:01:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:36 smithi104 ceph-mon[32726]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T13:01:37.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:37.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:37 smithi178 ceph-mon[33233]: from='client.? 172.21.15.178:0/1109431598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:01:38.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:37 smithi038 ceph-mon[29400]: from='client.? 172.21.15.178:0/1109431598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:01:38.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:37 smithi104 ceph-mon[32726]: from='client.? 172.21.15.178:0/1109431598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:01:38.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:38 smithi178 ceph-mon[33233]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T13:01:39.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:38 smithi038 ceph-mon[29400]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T13:01:39.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:38 smithi104 ceph-mon[32726]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T13:01:40.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:40 smithi178 ceph-mon[33233]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:01:41.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:40 smithi038 ceph-mon[29400]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:01:41.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:40 smithi104 ceph-mon[32726]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:01:42.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:42 smithi178 ceph-mon[33233]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:01:43.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:42 smithi038 ceph-mon[29400]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:01:43.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:42 smithi104 ceph-mon[32726]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:01:44.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:44 smithi178 ceph-mon[33233]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:01:44.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:01:44.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:44.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:44 smithi178 ceph-mon[33233]: Deploying daemon osd.11 on smithi178 2024-04-25T13:01:45.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:44 smithi038 ceph-mon[29400]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:01:45.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:01:45.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:45.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:44 smithi038 ceph-mon[29400]: Deploying daemon osd.11 on smithi178 2024-04-25T13:01:45.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:44 smithi104 ceph-mon[32726]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:01:45.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:01:45.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:45.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:44 smithi104 ceph-mon[32726]: Deploying daemon osd.11 on smithi178 2024-04-25T13:01:46.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:46 smithi178 ceph-mon[33233]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:01:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:46 smithi038 ceph-mon[29400]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:01:47.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:46 smithi104 ceph-mon[32726]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:01:48.868 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:48 smithi178 ceph-mon[33233]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:49.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:48 smithi038 ceph-mon[29400]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:49.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:48 smithi104 ceph-mon[32726]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:50 smithi038 ceph-mon[29400]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:50 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:50.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:50 smithi104 ceph-mon[32726]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:50.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:50.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:50.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:50 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:50.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:50 smithi178 ceph-mon[33233]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:50.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:50.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:50.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:50 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:52.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:52 smithi178 ceph-mon[33233]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:52.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:52 smithi038 ceph-mon[29400]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:52.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:52 smithi104 ceph-mon[32726]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:53.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.513+0000 7fb4d1ffb700 1 -- 172.21.15.178:0/3961368630 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb4d4062e80 con 0x7fb4c40a3cc0 2024-04-25T13:01:53.511 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 11 on host 'smithi178' 2024-04-25T13:01:53.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb4c40a3cc0 msgr2=0x7fb4c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:53.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb4c40a3cc0 0x7fb4c40a6180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4d41991d0 tx=0x7fb4c8008040 comp rx=0 tx=0).stop 2024-04-25T13:01:53.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 msgr2=0x7fb4d4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:53.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 0x7fb4d4198230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0006010 tx=0x7fb4c0009f90 comp rx=0 tx=0).stop 2024-04-25T13:01:53.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 shutdown_connections 2024-04-25T13:01:53.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb4c40a3cc0 0x7fb4c40a6180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:53.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4d4101c60 0x7fb4d4197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:53.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4102fb0 0x7fb4d419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:53.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 --2- 172.21.15.178:0/3961368630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d4102600 0x7fb4d4198230 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:53.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.515+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 >> 172.21.15.178:0/3961368630 conn(0x7fb4d40fc6f0 msgr2=0x7fb4d4105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:53.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.516+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 shutdown_connections 2024-04-25T13:01:53.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:01:53.516+0000 7fb4dadc9700 1 -- 172.21.15.178:0/3961368630 wait complete. 2024-04-25T13:01:53.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:53.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:53 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:54.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:53 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:01:54.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:54.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:53 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:01:54.143 DEBUG:teuthology.orchestra.run.smithi178:osd.11> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:01:54.146 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:01:54.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:01:54.402 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:54.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-mon[33233]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:54.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-mon[33233]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:01:54.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-mon[33233]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T13:01:54.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:54.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-mon[33233]: from='osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:01:54.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-mon[33233]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:01:54.739 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:01:54.653+0000 7f2792a94700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:01:54.739 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:01:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:01:54.665+0000 7f278e887700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:01:54.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:54 smithi038 ceph-mon[29400]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:54.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:54 smithi038 ceph-mon[29400]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:01:54.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:54 smithi038 ceph-mon[29400]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T13:01:54.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:54 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:54.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:54 smithi038 ceph-mon[29400]: from='osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:01:54.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:54 smithi038 ceph-mon[29400]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:01:55.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:54 smithi104 ceph-mon[32726]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:55.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:54 smithi104 ceph-mon[32726]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:01:55.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:54 smithi104 ceph-mon[32726]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T13:01:55.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:54 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:55.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:54 smithi104 ceph-mon[32726]: from='osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:01:55.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:54 smithi104 ceph-mon[32726]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-04-25T13:01:55.869 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:55 smithi038 ceph-mon[29400]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T13:01:55.869 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:55 smithi038 ceph-mon[29400]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:01:55.869 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:55 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:55.869 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:55 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:55.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:55 smithi178 ceph-mon[33233]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T13:01:55.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:55 smithi178 ceph-mon[33233]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:01:55.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:55 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:55.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:55 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:55 smithi104 ceph-mon[32726]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-04-25T13:01:56.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:55 smithi104 ceph-mon[32726]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:01:56.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:55 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:55 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 -- 172.21.15.38:0/2367337916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 msgr2=0x7f257010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 --2- 172.21.15.38:0/2367337916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f257010aef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f256800b0d0 tx=0x7f256800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 -- 172.21.15.38:0/2367337916 shutdown_connections 2024-04-25T13:01:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 --2- 172.21.15.38:0/2367337916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570102aa0 0x7f2570102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 --2- 172.21.15.38:0/2367337916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f257010aef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 --2- 172.21.15.38:0/2367337916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570103ca0 0x7f2570104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 -- 172.21.15.38:0/2367337916 >> 172.21.15.38:0/2367337916 conn(0x7f25700fe040 msgr2=0x7f2570100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 -- 172.21.15.38:0/2367337916 shutdown_connections 2024-04-25T13:01:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.294+0000 7f25774ec700 1 -- 172.21.15.38:0/2367337916 wait complete. 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 Processor -- start 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 -- start start 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 0x7f2570198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570103ca0 0x7f25701991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f25701a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f257010e6b0 con 0x7f2570104640 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f257010e530 con 0x7f2570103ca0 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f25774ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f257010e830 con 0x7f2570102aa0 2024-04-25T13:01:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 0x7f2570198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 0x7f2570198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.38:35762/0 (socket says 172.21.15.38:35762) 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575a89700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f25701a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 -- 172.21.15.38:0/970748439 learned_addr learned my addr 172.21.15.38:0/970748439 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2574a87700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570103ca0 0x7f25701991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 -- 172.21.15.38:0/970748439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570103ca0 msgr2=0x7f25701991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570103ca0 0x7f25701991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 -- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 msgr2=0x7f25701a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f25701a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575288700 1 -- 172.21.15.38:0/970748439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25701a0920 con 0x7f2570102aa0 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2574a87700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570103ca0 0x7f25701991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.295+0000 7f2575a89700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f25701a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.296+0000 7f2575288700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 0x7f2570198ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d980 tx=0x7f256c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.296+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c017030 con 0x7f2570102aa0 2024-04-25T13:01:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.296+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25701a0c10 con 0x7f2570102aa0 2024-04-25T13:01:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.296+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f256c0044f0 con 0x7f2570102aa0 2024-04-25T13:01:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.296+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c003aa0 con 0x7f2570102aa0 2024-04-25T13:01:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.296+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25701a11a0 con 0x7f2570102aa0 2024-04-25T13:01:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.297+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2570069730 con 0x7f2570102aa0 2024-04-25T13:01:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.298+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f256c01a070 con 0x7f2570102aa0 2024-04-25T13:01:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.299+0000 7f25667fc700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f255c0a3cc0 0x7f255c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.299+0000 7f2574a87700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f255c0a3cc0 0x7f255c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.299+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f256c0cfc60 con 0x7f2570102aa0 2024-04-25T13:01:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.299+0000 7f2574a87700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f255c0a3cc0 0x7f255c0a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f257019a1b0 tx=0x7f2560009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.302+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f256c09cac0 con 0x7f2570102aa0 2024-04-25T13:01:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.512+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f257002e090 con 0x7f2570102aa0 2024-04-25T13:01:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.512+0000 7f25667fc700 1 -- 172.21.15.38:0/970748439 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f256c09c3c0 con 0x7f2570102aa0 2024-04-25T13:01:56.510 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:01:56.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f255c0a3cc0 msgr2=0x7f255c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:56.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f255c0a3cc0 0x7f255c0a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f257019a1b0 tx=0x7f2560009260 comp rx=0 tx=0).stop 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 msgr2=0x7f2570198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 0x7f2570198ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d980 tx=0x7f256c00dc90 comp rx=0 tx=0).stop 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 shutdown_connections 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f255c0a3cc0 0x7f255c0a6180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2570104640 0x7f25701a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2570102aa0 0x7f2570198ca0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 --2- 172.21.15.38:0/970748439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2570103ca0 0x7f25701991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.514+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 >> 172.21.15.38:0/970748439 conn(0x7f25700fe040 msgr2=0x7f2570106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.515+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 shutdown_connections 2024-04-25T13:01:56.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:56.515+0000 7f25774ec700 1 -- 172.21.15.38:0/970748439 wait complete. 2024-04-25T13:01:56.812 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: purged_snaps scrub starts 2024-04-25T13:01:56.812 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: purged_snaps scrub ok 2024-04-25T13:01:56.812 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:56.812 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.812 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] boot 2024-04-25T13:01:56.812 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T13:01:56.813 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.813 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:56 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/970748439' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: purged_snaps scrub starts 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: purged_snaps scrub ok 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] boot 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:56.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:56 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/970748439' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:01:57.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: purged_snaps scrub starts 2024-04-25T13:01:57.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: purged_snaps scrub ok 2024-04-25T13:01:57.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:01:57.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:57.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: osd.11 [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] boot 2024-04-25T13:01:57.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T13:01:57.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:01:57.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:56 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/970748439' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:01:57.296 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714050115,"num_in_osds":12,"osd_in_since":1714050096,"num_remapped_pgs":0} 2024-04-25T13:01:57.296 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:01:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:01:57.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:01:57.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: Detected new or changed devices on smithi178 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi178 to 4272M 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:57.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:57 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: Detected new or changed devices on smithi178 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi178 to 4272M 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:57.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:57.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:57 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:01:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: Detected new or changed devices on smithi178 2024-04-25T13:01:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:58.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi178 to 4272M 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:01:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:57 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:01:59.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 -- 172.21.15.38:0/1525949376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f349406b1f0 msgr2=0x7f349406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 --2- 172.21.15.38:0/1525949376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f349406b1f0 0x7f349406b670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3484009990 tx=0x7f3484009ca0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 -- 172.21.15.38:0/1525949376 shutdown_connections 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 --2- 172.21.15.38:0/1525949376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3494108a60 0x7f3494108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 --2- 172.21.15.38:0/1525949376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 0x7f349410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 --2- 172.21.15.38:0/1525949376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f349406b1f0 0x7f349406b670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 -- 172.21.15.38:0/1525949376 >> 172.21.15.38:0/1525949376 conn(0x7f34940789e0 msgr2=0x7f3494078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 -- 172.21.15.38:0/1525949376 shutdown_connections 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.381+0000 7f349a812700 1 -- 172.21.15.38:0/1525949376 wait complete. 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 Processor -- start 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 -- start start 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f349406b1f0 0x7f3494199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 0x7f34941997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 0x7f349419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3494112d60 con 0x7f3494108a60 2024-04-25T13:01:59.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3494112be0 con 0x7f349406b1f0 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f349a812700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3494112ee0 con 0x7f349406bbb0 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f3498daf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 0x7f349419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f3498daf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 0x7f349419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58914/0 (socket says 172.21.15.38:58914) 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f3498daf700 1 -- 172.21.15.38:0/2946686809 learned_addr learned my addr 172.21.15.38:0/2946686809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.382+0000 7f3493fff700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f349406b1f0 0x7f3494199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f34937fe700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 0x7f34941997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f3498daf700 1 -- 172.21.15.38:0/2946686809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 msgr2=0x7f34941997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f3498daf700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 0x7f34941997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f3498daf700 1 -- 172.21.15.38:0/2946686809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f349406b1f0 msgr2=0x7f3494199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f3498daf700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f349406b1f0 0x7f3494199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f3498daf700 1 -- 172.21.15.38:0/2946686809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349419e2a0 con 0x7f3494108a60 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f34937fe700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 0x7f34941997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:01:59.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f3498daf700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 0x7f349419db60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f348c007ee0 tx=0x7f348c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:59.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c00dbd0 con 0x7f3494108a60 2024-04-25T13:01:59.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349419e590 con 0x7f3494108a60 2024-04-25T13:01:59.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f348c00dd70 con 0x7f3494108a60 2024-04-25T13:01:59.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c005a50 con 0x7f3494108a60 2024-04-25T13:01:59.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.383+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34941a5e10 con 0x7f3494108a60 2024-04-25T13:01:59.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.384+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3494069730 con 0x7f3494108a60 2024-04-25T13:01:59.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:01:59 smithi038 ceph-mon[29400]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:59.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.385+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f348c01a020 con 0x7f3494108a60 2024-04-25T13:01:59.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.386+0000 7f34917fa700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f347c0a3de0 0x7f347c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:01:59.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.386+0000 7f3493fff700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f347c0a3de0 0x7f347c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:01:59.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.386+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f348c0cf720 con 0x7f3494108a60 2024-04-25T13:01:59.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.387+0000 7f3493fff700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f347c0a3de0 0x7f347c0a62a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f348800abd0 tx=0x7f348800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:01:59.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.391+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f348c010070 con 0x7f3494108a60 2024-04-25T13:01:59.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:01:59 smithi178 ceph-mon[33233]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:59.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:01:59 smithi104 ceph-mon[32726]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:01:59.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.602+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f349402df80 con 0x7f3494108a60 2024-04-25T13:01:59.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.603+0000 7f34917fa700 1 -- 172.21.15.38:0/2946686809 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18409 (secure 0 0 0) 0x7f348c09c5a0 con 0x7f3494108a60 2024-04-25T13:01:59.601 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:01:59.601 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:01:56.666084+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:01:59.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.605+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f347c0a3de0 msgr2=0x7f347c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.605+0000 7f349a812700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f347c0a3de0 0x7f347c0a62a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f348800abd0 tx=0x7f348800a380 comp rx=0 tx=0).stop 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.605+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 msgr2=0x7f349419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.605+0000 7f349a812700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 0x7f349419db60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f348c007ee0 tx=0x7f348c00d3b0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 shutdown_connections 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f347c0a3de0 0x7f347c0a62a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f349406b1f0 0x7f3494199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3494108a60 0x7f349419db60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 --2- 172.21.15.38:0/2946686809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f349406bbb0 0x7f34941997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 >> 172.21.15.38:0/2946686809 conn(0x7f34940789e0 msgr2=0x7f34940ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 shutdown_connections 2024-04-25T13:01:59.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:01:59.606+0000 7f349a812700 1 -- 172.21.15.38:0/2946686809 wait complete. 2024-04-25T13:02:00.436 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:00 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2946686809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:00.438 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:02:00.438 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:02:00.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:00 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2946686809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:00.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:00 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2946686809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:00.689 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:01.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:01 smithi178 ceph-mon[33233]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:01.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:01 smithi038 ceph-mon[29400]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:01.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:01 smithi104 ceph-mon[32726]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:02.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:02 smithi038 ceph-mon[29400]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:02.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:02 smithi178 ceph-mon[33233]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:02.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 -- 172.21.15.38:0/607847606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0107990 msgr2=0x7f6da0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:02.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/607847606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0107990 0x7f6da0107d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009990 tx=0x7f6d9c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 -- 172.21.15.38:0/607847606 shutdown_connections 2024-04-25T13:02:02.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/607847606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6da0102300 0x7f6da010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/607847606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0107990 0x7f6da0107d70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/607847606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0101940 0x7f6da0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 -- 172.21.15.38:0/607847606 >> 172.21.15.38:0/607847606 conn(0x7f6da00788f0 msgr2=0x7f6da0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:02.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 -- 172.21.15.38:0/607847606 shutdown_connections 2024-04-25T13:02:02.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.492+0000 7f6da7bf9700 1 -- 172.21.15.38:0/607847606 wait complete. 2024-04-25T13:02:02.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 Processor -- start 2024-04-25T13:02:02.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 -- start start 2024-04-25T13:02:02.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 0x7f6da019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:02.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6da0102300 0x7f6da019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:02.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0107990 0x7f6da01a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:02.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0111cb0 con 0x7f6da0101940 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0111b30 con 0x7f6da0107990 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da7bf9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0111e30 con 0x7f6da0102300 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da5995700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 0x7f6da019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da5995700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 0x7f6da019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37476/0 (socket says 172.21.15.38:37476) 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da5995700 1 -- 172.21.15.38:0/3529341447 learned_addr learned my addr 172.21.15.38:0/3529341447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da6196700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0107990 0x7f6da01a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.493+0000 7f6da5194700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6da0102300 0x7f6da019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da5995700 1 -- 172.21.15.38:0/3529341447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6da0102300 msgr2=0x7f6da019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da5995700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6da0102300 0x7f6da019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da5995700 1 -- 172.21.15.38:0/3529341447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0107990 msgr2=0x7f6da01a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da5995700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0107990 0x7f6da01a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da5995700 1 -- 172.21.15.38:0/3529341447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da01a56b0 con 0x7f6da0101940 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da6196700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0107990 0x7f6da01a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:02.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da5995700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 0x7f6da019a0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:02.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c027070 con 0x7f6da0101940 2024-04-25T13:02:02.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da01a58e0 con 0x7f6da0101940 2024-04-25T13:02:02.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6d9c00fb40 con 0x7f6da0101940 2024-04-25T13:02:02.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c017650 con 0x7f6da0101940 2024-04-25T13:02:02.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.494+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da01a5d40 con 0x7f6da0101940 2024-04-25T13:02:02.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.495+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da0109eb0 con 0x7f6da0101940 2024-04-25T13:02:02.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.496+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6d9c02a070 con 0x7f6da0101940 2024-04-25T13:02:02.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.497+0000 7f6d96ffd700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6d8c0a3c40 0x7f6d8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:02.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.497+0000 7f6da5194700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6d8c0a3c40 0x7f6d8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:02.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.497+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6d9c0d1070 con 0x7f6da0101940 2024-04-25T13:02:02.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.498+0000 7f6da5194700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6d8c0a3c40 0x7f6d8c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d90006fd0 tx=0x7f6d90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:02.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.501+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d9c013070 con 0x7f6da0101940 2024-04-25T13:02:02.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:02 smithi104 ceph-mon[32726]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:02.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.708+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6da006ad70 con 0x7f6da0101940 2024-04-25T13:02:02.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.709+0000 7f6d96ffd700 1 -- 172.21.15.38:0/3529341447 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f6d9c09df80 con 0x7f6da0101940 2024-04-25T13:02:02.707 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:02:02.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6d8c0a3c40 msgr2=0x7f6d8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:02.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6d8c0a3c40 0x7f6d8c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d90006fd0 tx=0x7f6d90008040 comp rx=0 tx=0).stop 2024-04-25T13:02:02.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 msgr2=0x7f6da019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:02.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 0x7f6da019a0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c0047b0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 shutdown_connections 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6d8c0a3c40 0x7f6d8c0a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6da0107990 0x7f6da01a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da0101940 0x7f6da019a0f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 --2- 172.21.15.38:0/3529341447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6da0102300 0x7f6da019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 >> 172.21.15.38:0/3529341447 conn(0x7f6da00788f0 msgr2=0x7f6da01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 shutdown_connections 2024-04-25T13:02:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:02.711+0000 7f6da7bf9700 1 -- 172.21.15.38:0/3529341447 wait complete. 2024-04-25T13:02:03.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:03 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3529341447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.a on smithi038 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.d on smithi038 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.g on smithi038 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.j on smithi038 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.b on smithi104 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.e on smithi104 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.h on smithi104 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.k on smithi104 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.c on smithi178 2024-04-25T13:02:03.482 INFO:tasks.cephadm:Adding mds.f on smithi178 2024-04-25T13:02:03.483 INFO:tasks.cephadm:Adding mds.i on smithi178 2024-04-25T13:02:03.483 INFO:tasks.cephadm:Adding mds.l on smithi178 2024-04-25T13:02:03.483 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l' 2024-04-25T13:02:03.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:03 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3529341447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:03.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:03 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3529341447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:03.733 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:02:04.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:04 smithi178 ceph-mon[33233]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:04.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:04 smithi038 ceph-mon[29400]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:04.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:04 smithi104 ceph-mon[32726]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 -- 172.21.15.178:0/3923713511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418102ff0 msgr2=0x7f9418103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 --2- 172.21.15.178:0/3923713511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418102ff0 0x7f9418103410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94080099c0 tx=0x7f9408009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 -- 172.21.15.178:0/3923713511 shutdown_connections 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 --2- 172.21.15.178:0/3923713511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 0x7f9418109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 --2- 172.21.15.178:0/3923713511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418102ff0 0x7f9418103410 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 --2- 172.21.15.178:0/3923713511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418101d30 0x7f9418102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 -- 172.21.15.178:0/3923713511 >> 172.21.15.178:0/3923713511 conn(0x7f94180fc780 msgr2=0x7f94180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 -- 172.21.15.178:0/3923713511 shutdown_connections 2024-04-25T13:02:05.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 -- 172.21.15.178:0/3923713511 wait complete. 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 Processor -- start 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.396+0000 7f941e9e9700 1 -- start start 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941e9e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 0x7f9418101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941e9e9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 0x7f941819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941e9e9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418102ff0 0x7f94181a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941e9e9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f941810d6b0 con 0x7f94181026d0 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941e9e9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f941810d530 con 0x7f9418102ff0 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941e9e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f941810d830 con 0x7f9418101d30 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 0x7f9418101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 0x7f9418101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43374/0 (socket says 172.21.15.178:43374) 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 -- 172.21.15.178:0/791350973 learned_addr learned my addr 172.21.15.178:0/791350973 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f94177fe700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 0x7f941819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941cf86700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418102ff0 0x7f94181a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:05.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 -- 172.21.15.178:0/791350973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418102ff0 msgr2=0x7f94181a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418102ff0 0x7f94181a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 -- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 msgr2=0x7f941819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 0x7f941819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 -- 172.21.15.178:0/791350973 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a0a00 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f941cf86700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418102ff0 0x7f94181a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f94177fe700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 0x7f941819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f9417fff700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 0x7f9418101780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9408006010 tx=0x7f94080047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.397+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408027070 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.398+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9408004110 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.398+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940800f460 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.398+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a0c90 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.398+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a1140 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.398+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9418069730 con 0x7f9418101d30 2024-04-25T13:02:05.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.400+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f940802a070 con 0x7f9418101d30 2024-04-25T13:02:05.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.400+0000 7f94157fa700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94000a3cc0 0x7f94000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:05.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.401+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f94080d0da0 con 0x7f9418101d30 2024-04-25T13:02:05.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.401+0000 7f94177fe700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94000a3cc0 0x7f94000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:05.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.401+0000 7f94177fe700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94000a3cc0 0x7f94000a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9418101d00 tx=0x7f940c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:05.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.405+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f940805eff0 con 0x7f9418101d30 2024-04-25T13:02:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.614+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}) -- 0x7f9418063140 con 0x7f94000a3cc0 2024-04-25T13:02:05.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.623+0000 7f94157fa700 1 -- 172.21.15.178:0/791350973 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9418063140 con 0x7f94000a3cc0 2024-04-25T13:02:05.622 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mds.all update... 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94000a3cc0 msgr2=0x7f94000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94000a3cc0 0x7f94000a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9418101d00 tx=0x7f940c00b2c0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 msgr2=0x7f9418101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 0x7f9418101780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9408006010 tx=0x7f94080047b0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 shutdown_connections 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94000a3cc0 0x7f94000a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9418102ff0 0x7f94181a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418101d30 0x7f9418101780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 --2- 172.21.15.178:0/791350973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94181026d0 0x7f941819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 >> 172.21.15.178:0/791350973 conn(0x7f94180fc780 msgr2=0x7f9418105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 shutdown_connections 2024-04-25T13:02:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-25T13:02:05.626+0000 7f941e9e9700 1 -- 172.21.15.178:0/791350973 wait complete. 2024-04-25T13:02:06.323 DEBUG:teuthology.orchestra.run.smithi038:mds.a> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:02:06.326 DEBUG:teuthology.orchestra.run.smithi038:mds.d> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:02:06.370 DEBUG:teuthology.orchestra.run.smithi038:mds.g> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:02:06.413 DEBUG:teuthology.orchestra.run.smithi038:mds.j> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:02:06.455 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:02:06.458 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:02:06.500 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:02:06.542 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:02:06.585 DEBUG:teuthology.orchestra.run.smithi178:mds.c> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:02:06.588 DEBUG:teuthology.orchestra.run.smithi178:mds.f> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:02:06.631 DEBUG:teuthology.orchestra.run.smithi178:mds.i> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:02:06.674 DEBUG:teuthology.orchestra.run.smithi178:mds.l> sudo journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:02:06.717 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:02:06.717 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:06.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:06.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:06.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-04-25T13:02:06.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:06 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:06.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 -- 172.21.15.38:0/1567492231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac103b70 msgr2=0x7f9fac10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 --2- 172.21.15.38:0/1567492231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac103b70 0x7f9fac10a400 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009ca0 tx=0x7f9fa402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 -- 172.21.15.38:0/1567492231 shutdown_connections 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 --2- 172.21.15.38:0/1567492231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac103b70 0x7f9fac10a400 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 --2- 172.21.15.38:0/1567492231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fac1031b0 0x7f9fac103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 --2- 172.21.15.38:0/1567492231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fac101fc0 0x7f9fac1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 -- 172.21.15.38:0/1567492231 >> 172.21.15.38:0/1567492231 conn(0x7f9fac0fd550 msgr2=0x7f9fac0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 -- 172.21.15.38:0/1567492231 shutdown_connections 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.861+0000 7f9fb1220640 1 -- 172.21.15.38:0/1567492231 wait complete. 2024-04-25T13:02:06.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 Processor -- start 2024-04-25T13:02:06.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 -- start start 2024-04-25T13:02:06.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 0x7f9fac1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fac1031b0 0x7f9fac198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fac103b70 0x7f9fac19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10dc20 con 0x7f9fac101fc0 2024-04-25T13:02:06.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10daa0 con 0x7f9fac103b70 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fb1220640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10dda0 con 0x7f9fac1031b0 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9faad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 0x7f9fac1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9faad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 0x7f9fac1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37498/0 (socket says 172.21.15.38:37498) 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9faad9d640 1 -- 172.21.15.38:0/4147452158 learned_addr learned my addr 172.21.15.38:0/4147452158 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9faa59c640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fac1031b0 0x7f9fac198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.862+0000 7f9fab59e640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fac103b70 0x7f9fac19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9faad9d640 1 -- 172.21.15.38:0/4147452158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fac1031b0 msgr2=0x7f9fac198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9faad9d640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fac1031b0 0x7f9fac198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9faad9d640 1 -- 172.21.15.38:0/4147452158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fac103b70 msgr2=0x7f9fac19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9faad9d640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fac103b70 0x7f9fac19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9faad9d640 1 -- 172.21.15.38:0/4147452158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac19fef0 con 0x7f9fac101fc0 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9faad9d640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 0x7f9fac1982d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e9c0 tx=0x7f9f9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:06.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400cd80 con 0x7f9fac101fc0 2024-04-25T13:02:06.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1a01e0 con 0x7f9fac101fc0 2024-04-25T13:02:06.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f940035f0 con 0x7f9fac101fc0 2024-04-25T13:02:06.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94010690 con 0x7f9fac101fc0 2024-04-25T13:02:06.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.863+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1a0730 con 0x7f9fac101fc0 2024-04-25T13:02:06.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.864+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac109280 con 0x7f9fac101fc0 2024-04-25T13:02:06.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.865+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f94003b30 con 0x7f9fac101fc0 2024-04-25T13:02:06.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.866+0000 7f9fa3fff640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f980a3c30 0x7f9f980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:06.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.866+0000 7f9faa59c640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f980a3c30 0x7f9f980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:06.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.866+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9f940d02e0 con 0x7f9fac101fc0 2024-04-25T13:02:06.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.866+0000 7f9faa59c640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f980a3c30 0x7f9f980a60f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9fac1997e0 tx=0x7f9f9c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:06.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:06.870+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f9409d0c0 con 0x7f9fac101fc0 2024-04-25T13:02:06.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:06.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:06.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:06 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:07.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.058+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fac062b50 con 0x7f9fac101fc0 2024-04-25T13:02:07.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.059+0000 7f9fa3fff640 1 -- 172.21.15.38:0/4147452158 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18409 (secure 0 0 0) 0x7f9f9409c9c0 con 0x7f9fac101fc0 2024-04-25T13:02:07.057 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:07.057 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:01:56.666084+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.060+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f980a3c30 msgr2=0x7f9f980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.060+0000 7f9fb1220640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f980a3c30 0x7f9f980a60f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9fac1997e0 tx=0x7f9f9c00b390 comp rx=0 tx=0).stop 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.060+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 msgr2=0x7f9fac1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.060+0000 7f9fb1220640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 0x7f9fac1982d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e9c0 tx=0x7f9f9400ee80 comp rx=0 tx=0).stop 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 shutdown_connections 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f980a3c30 0x7f9f980a60f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fac103b70 0x7f9fac19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fac1031b0 0x7f9fac198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 --2- 172.21.15.38:0/4147452158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fac101fc0 0x7f9fac1982d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 >> 172.21.15.38:0/4147452158 conn(0x7f9fac0fd550 msgr2=0x7f9fac106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 shutdown_connections 2024-04-25T13:02:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.061+0000 7f9fb1220640 1 -- 172.21.15.38:0/4147452158 wait complete. 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:07.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:06 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:07.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:02:07.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:02:07.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 -- 172.21.15.38:0/2643863651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214100e90 msgr2=0x7fc2141012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 --2- 172.21.15.38:0/2643863651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214100e90 0x7fc2141012b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc204009900 tx=0x7fc20402a410 comp rx=0 tx=0).stop 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 -- 172.21.15.38:0/2643863651 shutdown_connections 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 --2- 172.21.15.38:0/2643863651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2141029a0 0x7fc2141092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 --2- 172.21.15.38:0/2643863651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc214101fe0 0x7fc214102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 --2- 172.21.15.38:0/2643863651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214100e90 0x7fc2141012b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 -- 172.21.15.38:0/2643863651 >> 172.21.15.38:0/2643863651 conn(0x7fc2140fc400 msgr2=0x7fc2140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 -- 172.21.15.38:0/2643863651 shutdown_connections 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 -- 172.21.15.38:0/2643863651 wait complete. 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 Processor -- start 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.179+0000 7fc21a720640 1 -- start start 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc214100e90 0x7fc214069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 0x7fc21419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141029a0 0x7fc21419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc21410cb00 con 0x7fc214101fe0 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc21410c980 con 0x7fc214100e90 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc21410cc80 con 0x7fc2141029a0 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 0x7fc21419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 0x7fc21419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37514/0 (socket says 172.21.15.38:37514) 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 -- 172.21.15.38:0/1640477210 learned_addr learned my addr 172.21.15.38:0/1640477210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc213fff640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc214100e90 0x7fc214069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc218cbd640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141029a0 0x7fc21419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 -- 172.21.15.38:0/1640477210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141029a0 msgr2=0x7fc21419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141029a0 0x7fc21419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 -- 172.21.15.38:0/1640477210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc214100e90 msgr2=0x7fc214069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc214100e90 0x7fc214069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 -- 172.21.15.38:0/1640477210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21419ff00 con 0x7fc214101fe0 2024-04-25T13:02:07.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2137fe640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 0x7fc21419d000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e9c0 tx=0x7fc20800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800ccc0 con 0x7fc214101fe0 2024-04-25T13:02:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc2080035f0 con 0x7fc214101fe0 2024-04-25T13:02:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2141a01f0 con 0x7fc214101fe0 2024-04-25T13:02:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800ce60 con 0x7fc214101fe0 2024-04-25T13:02:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.180+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2141a0740 con 0x7fc214101fe0 2024-04-25T13:02:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.182+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2141082c0 con 0x7fc214101fe0 2024-04-25T13:02:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.183+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc2080037a0 con 0x7fc214101fe0 2024-04-25T13:02:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.183+0000 7fc2117fa640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc1fc0a3bb0 0x7fc1fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.183+0000 7fc213fff640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc1fc0a3bb0 0x7fc1fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.183+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2080d02f0 con 0x7fc214101fe0 2024-04-25T13:02:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.184+0000 7fc213fff640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc1fc0a3bb0 0x7fc1fc0a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc204000c00 tx=0x7fc204031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.188+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc20809d170 con 0x7fc214101fe0 2024-04-25T13:02:07.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.374+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc21402d810 con 0x7fc214101fe0 2024-04-25T13:02:07.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.375+0000 7fc2117fa640 1 -- 172.21.15.38:0/1640477210 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc20809ca70 con 0x7fc214101fe0 2024-04-25T13:02:07.373 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc1fc0a3bb0 msgr2=0x7fc1fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc1fc0a3bb0 0x7fc1fc0a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc204000c00 tx=0x7fc204031510 comp rx=0 tx=0).stop 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 msgr2=0x7fc21419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 0x7fc21419d000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e9c0 tx=0x7fc20800edc0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 shutdown_connections 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc1fc0a3bb0 0x7fc1fc0a6070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141029a0 0x7fc21419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc214101fe0 0x7fc21419d000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 --2- 172.21.15.38:0/1640477210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc214100e90 0x7fc214069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 >> 172.21.15.38:0/1640477210 conn(0x7fc2140fc400 msgr2=0x7fc214105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:07.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 shutdown_connections 2024-04-25T13:02:07.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.377+0000 7fc21a720640 1 -- 172.21.15.38:0/1640477210 wait complete. 2024-04-25T13:02:07.386 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:07.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 -- 172.21.15.38:0/3561525780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 msgr2=0x7fca4410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/3561525780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4410a0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009ca0 tx=0x7fca3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 -- 172.21.15.38:0/3561525780 shutdown_connections 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/3561525780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4410a0e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/3561525780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca441053c0 0x7fca441077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/3561525780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca44102a90 0x7fca44104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 -- 172.21.15.38:0/3561525780 >> 172.21.15.38:0/3561525780 conn(0x7fca440fc440 msgr2=0x7fca440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 -- 172.21.15.38:0/3561525780 shutdown_connections 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.489+0000 7fca4a8c2640 1 -- 172.21.15.38:0/3561525780 wait complete. 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 Processor -- start 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 -- start start 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca44102a90 0x7fca441981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca441053c0 0x7fca44198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca4410dc10 con 0x7fca44107cf0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca4410da90 con 0x7fca44102a90 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca4a8c2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca4410dd90 con 0x7fca441053c0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca48e5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca48e5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37540/0 (socket says 172.21.15.38:37540) 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca48e5f640 1 -- 172.21.15.38:0/2984174035 learned_addr learned my addr 172.21.15.38:0/2984174035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca437fe640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca441053c0 0x7fca44198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.490+0000 7fca43fff640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca44102a90 0x7fca441981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca48e5f640 1 -- 172.21.15.38:0/2984174035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca441053c0 msgr2=0x7fca44198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca48e5f640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca441053c0 0x7fca44198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca48e5f640 1 -- 172.21.15.38:0/2984174035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca44102a90 msgr2=0x7fca441981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca48e5f640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca44102a90 0x7fca441981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca48e5f640 1 -- 172.21.15.38:0/2984174035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4419fdf0 con 0x7fca44107cf0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca437fe640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca441053c0 0x7fca44198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca48e5f640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4419f780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009dc0 tx=0x7fca3c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:07.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003bf0 con 0x7fca44107cf0 2024-04-25T13:02:07.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca441a0080 con 0x7fca44107cf0 2024-04-25T13:02:07.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fca3c003d90 con 0x7fca44107cf0 2024-04-25T13:02:07.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c02fc00 con 0x7fca44107cf0 2024-04-25T13:02:07.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.491+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca441a0620 con 0x7fca44107cf0 2024-04-25T13:02:07.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.492+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca440fdf30 con 0x7fca44107cf0 2024-04-25T13:02:07.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.493+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fca3c0046c0 con 0x7fca44107cf0 2024-04-25T13:02:07.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.494+0000 7fca417fa640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fca2c0a3bb0 0x7fca2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:07.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.494+0000 7fca43fff640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fca2c0a3bb0 0x7fca2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:07.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.494+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fca3c0f1620 con 0x7fca44107cf0 2024-04-25T13:02:07.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.494+0000 7fca43fff640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fca2c0a3bb0 0x7fca2c0a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca38009bd0 tx=0x7fca38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:07.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.498+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca3c0be400 con 0x7fca44107cf0 2024-04-25T13:02:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.725+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca44062b50 con 0x7fca44107cf0 2024-04-25T13:02:07.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.725+0000 7fca417fa640 1 -- 172.21.15.38:0/2984174035 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fca3c0bdd00 con 0x7fca44107cf0 2024-04-25T13:02:07.723 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:07.724 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:02:07.724 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 1 2024-04-25T13:02:07.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fca2c0a3bb0 msgr2=0x7fca2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fca2c0a3bb0 0x7fca2c0a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca38009bd0 tx=0x7fca38005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 msgr2=0x7fca4419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4419f780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009dc0 tx=0x7fca3c031840 comp rx=0 tx=0).stop 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 shutdown_connections 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fca2c0a3bb0 0x7fca2c0a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca44107cf0 0x7fca4419f780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca441053c0 0x7fca44198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 --2- 172.21.15.38:0/2984174035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca44102a90 0x7fca441981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.727+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 >> 172.21.15.38:0/2984174035 conn(0x7fca440fc440 msgr2=0x7fca441033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.728+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 shutdown_connections 2024-04-25T13:02:07.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:07.728+0000 7fca4a8c2640 1 -- 172.21.15.38:0/2984174035 wait complete. 2024-04-25T13:02:07.735 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:07 smithi038 ceph-mon[29400]: Deploying daemon mds.l on smithi178 2024-04-25T13:02:07.735 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:07 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/4147452158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:07.735 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:07 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1640477210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:07 smithi178 ceph-mon[33233]: Deploying daemon mds.l on smithi178 2024-04-25T13:02:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:07 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/4147452158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:07.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:07 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1640477210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:08.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:07 smithi104 ceph-mon[32726]: Deploying daemon mds.l on smithi178 2024-04-25T13:02:08.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:07 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/4147452158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:08.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:07 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1640477210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:08.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:08 smithi178 ceph-mon[33233]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:08.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:08 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2984174035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:09.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:08 smithi038 ceph-mon[29400]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:09.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:08 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2984174035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:09.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:08 smithi104 ceph-mon[32726]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:09.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:08 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2984174035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:09.488 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:02:09 smithi178 systemd[1]: Started Ceph mds.l for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:09.737 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:09.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 -- 172.21.15.38:0/3405005183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8103b70 msgr2=0x7f26e810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3405005183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8103b70 0x7f26e810a400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009ca0 tx=0x7f26e002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 -- 172.21.15.38:0/3405005183 shutdown_connections 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3405005183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8103b70 0x7f26e810a400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3405005183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 0x7f26e8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3405005183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26e8101fc0 0x7f26e81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 -- 172.21.15.38:0/3405005183 >> 172.21.15.38:0/3405005183 conn(0x7f26e80fd550 msgr2=0x7f26e80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 -- 172.21.15.38:0/3405005183 shutdown_connections 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.854+0000 7f26eca8a640 1 -- 172.21.15.38:0/3405005183 wait complete. 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 Processor -- start 2024-04-25T13:02:09.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 -- start start 2024-04-25T13:02:09.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 0x7f26e8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:09.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 0x7f26e8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:09.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26e8103b70 0x7f26e819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810dc00 con 0x7f26e8101fc0 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810da80 con 0x7f26e81031b0 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26eca8a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810dd80 con 0x7f26e8103b70 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 0x7f26e8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 0x7f26e8198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32826/0 (socket says 172.21.15.38:32826) 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 -- 172.21.15.38:0/3679195231 learned_addr learned my addr 172.21.15.38:0/3679195231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26dffff640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 0x7f26e8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e6d9d640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26e8103b70 0x7f26e819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 -- 172.21.15.38:0/3679195231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26e8103b70 msgr2=0x7f26e819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26e8103b70 0x7f26e819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 -- 172.21.15.38:0/3679195231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 msgr2=0x7f26e8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 0x7f26e8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:09.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26e659c640 1 -- 172.21.15.38:0/3679195231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e819fe70 con 0x7f26e8101fc0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.855+0000 7f26dffff640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 0x7f26e8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.856+0000 7f26e659c640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 0x7f26e8198250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26d800d770 tx=0x7f26d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.856+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8017070 con 0x7f26e8101fc0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.856+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81a0160 con 0x7f26e8101fc0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.856+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f26d8009c30 con 0x7f26e8101fc0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.856+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8003d70 con 0x7f26e8101fc0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.856+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81a0700 con 0x7f26e8101fc0 2024-04-25T13:02:09.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.857+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e81092e0 con 0x7f26e8101fc0 2024-04-25T13:02:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.858+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26d8004000 con 0x7f26e8101fc0 2024-04-25T13:02:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.859+0000 7f26df7fe640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26d40a3bb0 0x7f26d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.859+0000 7f26dffff640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26d40a3bb0 0x7f26d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.859+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f26d80cfc70 con 0x7f26e8101fc0 2024-04-25T13:02:09.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.859+0000 7f26dffff640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26d40a3bb0 0x7f26d40a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26e8199760 tx=0x7f26d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:09.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:09.862+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26d809caf0 con 0x7f26e8101fc0 2024-04-25T13:02:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.090+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26e802d5c0 con 0x7f26e8101fc0 2024-04-25T13:02:10.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.093+0000 7f26df7fe640 1 -- 172.21.15.38:0/3679195231 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f26d8009de0 con 0x7f26e8101fc0 2024-04-25T13:02:10.092 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:10.092 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:02:10.092 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-04-25T13:02:10.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26d40a3bb0 msgr2=0x7f26d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26d40a3bb0 0x7f26d40a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26e8199760 tx=0x7f26d0005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 msgr2=0x7f26e8198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 0x7f26e8198250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26d800d770 tx=0x7f26d800dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 shutdown_connections 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26d40a3bb0 0x7f26d40a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26e8103b70 0x7f26e819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26e81031b0 0x7f26e8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 --2- 172.21.15.38:0/3679195231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e8101fc0 0x7f26e8198250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 >> 172.21.15.38:0/3679195231 conn(0x7f26e80fd550 msgr2=0x7f26e81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 shutdown_connections 2024-04-25T13:02:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:10.095+0000 7f26eca8a640 1 -- 172.21.15.38:0/3679195231 wait complete. 2024-04-25T13:02:10.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6832/2494647672,v1:172.21.15.178:6833/2494647672] up:boot 2024-04-25T13:02:10.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: fsmap 1 up:standby 2024-04-25T13:02:10.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:02:10.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:10.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:10.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:10 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6832/2494647672,v1:172.21.15.178:6833/2494647672] up:boot 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: fsmap 1 up:standby 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:10 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6832/2494647672,v1:172.21.15.178:6833/2494647672] up:boot 2024-04-25T13:02:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: fsmap 1 up:standby 2024-04-25T13:02:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:02:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:10.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:10.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:10.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:10 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:11.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:11 smithi104 ceph-mon[32726]: Deploying daemon mds.k on smithi104 2024-04-25T13:02:11.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:11 smithi104 ceph-mon[32726]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:11.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:11 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3679195231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:11.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:11 smithi178 ceph-mon[33233]: Deploying daemon mds.k on smithi104 2024-04-25T13:02:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:11 smithi178 ceph-mon[33233]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:11 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3679195231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:11.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:11 smithi038 ceph-mon[29400]: Deploying daemon mds.k on smithi104 2024-04-25T13:02:11.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:11 smithi038 ceph-mon[29400]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:11.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:11 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3679195231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:11.573 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:11 smithi104 systemd[1]: Starting Ceph mds.k for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:11.990 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:11 smithi104 podman[62306]: 2024-04-25 13:02:11.603940013 +0000 UTC m=+0.017792080 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:11.991 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:11 smithi104 podman[62306]: 2024-04-25 13:02:11.731245984 +0000 UTC m=+0.145098082 container create 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:02:11.991 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:11 smithi104 podman[62306]: 2024-04-25 13:02:11.992325394 +0000 UTC m=+0.406177446 container init 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T13:02:12.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:12.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3808290215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc101fa0 msgr2=0x7f57bc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:12.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3808290215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc101fa0 0x7f57bc1023c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009900 tx=0x7f57ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:12.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3808290215 shutdown_connections 2024-04-25T13:02:12.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3808290215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57bc103b50 0x7f57bc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3808290215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57bc103190 0x7f57bc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3808290215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc101fa0 0x7f57bc1023c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3808290215 >> 172.21.15.38:0/3808290215 conn(0x7f57bc0fd530 msgr2=0x7f57bc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3808290215 shutdown_connections 2024-04-25T13:02:12.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.227+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3808290215 wait complete. 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 Processor -- start 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 -- start start 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57bc101fa0 0x7f57bc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 0x7f57bc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57bc103b50 0x7f57bc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10dc00 con 0x7f57bc103190 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10da80 con 0x7f57bc103b50 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57c1fcd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10dd80 con 0x7f57bc101fa0 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57baffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 0x7f57bc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57baffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 0x7f57bc074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32852/0 (socket says 172.21.15.38:32852) 2024-04-25T13:02:12.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57baffd640 1 -- 172.21.15.38:0/3972852258 learned_addr learned my addr 172.21.15.38:0/3972852258 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57bbfff640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57bc103b50 0x7f57bc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.228+0000 7f57bb7fe640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57bc101fa0 0x7f57bc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57baffd640 1 -- 172.21.15.38:0/3972852258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57bc101fa0 msgr2=0x7f57bc075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57baffd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57bc101fa0 0x7f57bc075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57baffd640 1 -- 172.21.15.38:0/3972852258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57bc103b50 msgr2=0x7f57bc074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57baffd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57bc103b50 0x7f57bc074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57baffd640 1 -- 172.21.15.38:0/3972852258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc075110 con 0x7f57bc103190 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57baffd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 0x7f57bc074340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57a400d950 tx=0x7f57a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4017070 con 0x7f57bc103190 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f57a4009c30 con 0x7f57bc103190 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a400f6b0 con 0x7f57bc103190 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57bc1a23b0 con 0x7f57bc103190 2024-04-25T13:02:12.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.229+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57bc1a2900 con 0x7f57bc103190 2024-04-25T13:02:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.230+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57bc1092c0 con 0x7f57bc103190 2024-04-25T13:02:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.231+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f57a40109c0 con 0x7f57bc103190 2024-04-25T13:02:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.232+0000 7f57b8ff9640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f57a80a3bb0 0x7f57a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.232+0000 7f57bb7fe640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f57a80a3bb0 0x7f57a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.232+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f57a40cfb80 con 0x7f57bc103190 2024-04-25T13:02:12.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.233+0000 7f57bb7fe640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f57a80a3bb0 0x7f57a80a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009230 tx=0x7f57ac031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.237+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57a4009de0 con 0x7f57bc103190 2024-04-25T13:02:12.323 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:11 smithi104 podman[62306]: 2024-04-25 13:02:11.995379156 +0000 UTC m=+0.409231209 container start 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:02:12.323 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:12 smithi104 ceph-mds[62326]: -- 172.21.15.104:0/1764796080 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e953d11a40 con 0x55e953ce4800 2024-04-25T13:02:12.323 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:12 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k[62322]: starting mds.k at 2024-04-25T13:02:12.323 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:12 smithi104 bash[62306]: 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 2024-04-25T13:02:12.324 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:02:12 smithi104 systemd[1]: Started Ceph mds.k for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.465+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57bc02d5c0 con 0x7f57bc103190 2024-04-25T13:02:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.465+0000 7f57b8ff9640 1 -- 172.21.15.38:0/3972852258 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f57a40cf980 con 0x7f57bc103190 2024-04-25T13:02:12.464 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:12.464 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:02:12.465 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2024-04-25T13:02:12.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f57a80a3bb0 msgr2=0x7f57a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:12.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f57a80a3bb0 0x7f57a80a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009230 tx=0x7f57ac031510 comp rx=0 tx=0).stop 2024-04-25T13:02:12.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 msgr2=0x7f57bc074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 0x7f57bc074340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57a400d950 tx=0x7f57a400de10 comp rx=0 tx=0).stop 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 shutdown_connections 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f57a80a3bb0 0x7f57a80a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57bc103b50 0x7f57bc074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57bc103190 0x7f57bc074340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 --2- 172.21.15.38:0/3972852258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57bc101fa0 0x7f57bc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 >> 172.21.15.38:0/3972852258 conn(0x7f57bc0fd530 msgr2=0x7f57bc106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 shutdown_connections 2024-04-25T13:02:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:12.467+0000 7f57c1fcd640 1 -- 172.21.15.38:0/3972852258 wait complete. 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6832/650369302,v1:172.21.15.104:6833/650369302] up:boot 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: fsmap 2 up:standby 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:13.404 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:13.405 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:13 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3972852258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6832/650369302,v1:172.21.15.104:6833/650369302] up:boot 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: fsmap 2 up:standby 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:13 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3972852258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:13.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:13.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6832/650369302,v1:172.21.15.104:6833/650369302] up:boot 2024-04-25T13:02:13.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: fsmap 2 up:standby 2024-04-25T13:02:13.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:02:13.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:13.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:13.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:13.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:13.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:13 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3972852258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:14.265 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:14 smithi038 ceph-mon[29400]: Deploying daemon mds.j on smithi038 2024-04-25T13:02:14.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:14.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:14 smithi178 ceph-mon[33233]: Deploying daemon mds.j on smithi038 2024-04-25T13:02:14.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:14 smithi104 ceph-mon[32726]: Deploying daemon mds.j on smithi038 2024-04-25T13:02:14.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 -- 172.21.15.38:0/1107045645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 msgr2=0x7f8ff41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1107045645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff41092c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009ca0 tx=0x7f8fec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 -- 172.21.15.38:0/1107045645 shutdown_connections 2024-04-25T13:02:14.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1107045645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff41092c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1107045645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff4101fe0 0x7f8ff4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1107045645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ff4100e90 0x7f8ff41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 -- 172.21.15.38:0/1107045645 >> 172.21.15.38:0/1107045645 conn(0x7f8ff40fc400 msgr2=0x7f8ff40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 -- 172.21.15.38:0/1107045645 shutdown_connections 2024-04-25T13:02:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.594+0000 7f8ff975d640 1 -- 172.21.15.38:0/1107045645 wait complete. 2024-04-25T13:02:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 Processor -- start 2024-04-25T13:02:14.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 -- start start 2024-04-25T13:02:14.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff4100e90 0x7f8ff4104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ff4101fe0 0x7f8ff4103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff4103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410cb10 con 0x7f8ff41029a0 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410c990 con 0x7f8ff4101fe0 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff975d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410cc90 con 0x7f8ff4100e90 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff4103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff4103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32868/0 (socket says 172.21.15.38:32868) 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 -- 172.21.15.38:0/1066884412 learned_addr learned my addr 172.21.15.38:0/1066884412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff27fc640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ff4101fe0 0x7f8ff4103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff2ffd640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff4100e90 0x7f8ff4104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 -- 172.21.15.38:0/1066884412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff4100e90 msgr2=0x7f8ff4104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff4100e90 0x7f8ff4104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 -- 172.21.15.38:0/1066884412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ff4101fe0 msgr2=0x7f8ff4103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ff4101fe0 0x7f8ff4103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.595+0000 7f8ff37fe640 1 -- 172.21.15.38:0/1066884412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff4104220 con 0x7f8ff41029a0 2024-04-25T13:02:14.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.596+0000 7f8ff37fe640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff4103990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009dc0 tx=0x7f8fec003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.596+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec003d90 con 0x7f8ff41029a0 2024-04-25T13:02:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.596+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8fec004510 con 0x7f8ff41029a0 2024-04-25T13:02:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.596+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41a2360 con 0x7f8ff41029a0 2024-04-25T13:02:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.596+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec02fe90 con 0x7f8ff41029a0 2024-04-25T13:02:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.596+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a28b0 con 0x7f8ff41029a0 2024-04-25T13:02:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.597+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff41a2b10 con 0x7f8ff41029a0 2024-04-25T13:02:14.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.599+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fec0046c0 con 0x7f8ff41029a0 2024-04-25T13:02:14.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.599+0000 7f8fdbfff640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fdc0a3bb0 0x7f8fdc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:14.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.600+0000 7f8ff2ffd640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fdc0a3bb0 0x7f8fdc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:14.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.600+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8fec0f1980 con 0x7f8ff41029a0 2024-04-25T13:02:14.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.600+0000 7f8ff2ffd640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fdc0a3bb0 0x7f8fdc0a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005fd0 tx=0x7f8fe8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:14.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.603+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fec0be760 con 0x7f8ff41029a0 2024-04-25T13:02:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.839+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ff41a2e00 con 0x7f8ff41029a0 2024-04-25T13:02:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.839+0000 7f8fdbfff640 1 -- 172.21.15.38:0/1066884412 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f8fec0098b0 con 0x7f8ff41029a0 2024-04-25T13:02:14.839 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:14.839 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:02:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fdc0a3bb0 msgr2=0x7f8fdc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fdc0a3bb0 0x7f8fdc0a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005fd0 tx=0x7f8fe8005850 comp rx=0 tx=0).stop 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 msgr2=0x7f8ff4103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff4103990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009dc0 tx=0x7f8fec003970 comp rx=0 tx=0).stop 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 shutdown_connections 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fdc0a3bb0 0x7f8fdc0a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff41029a0 0x7f8ff4103990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ff4101fe0 0x7f8ff4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 --2- 172.21.15.38:0/1066884412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff4100e90 0x7f8ff4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 >> 172.21.15.38:0/1066884412 conn(0x7f8ff40fc400 msgr2=0x7f8ff40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 shutdown_connections 2024-04-25T13:02:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:14.842+0000 7f8ff975d640 1 -- 172.21.15.38:0/1066884412 wait complete. 2024-04-25T13:02:15.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:15 smithi038 ceph-mon[29400]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:15.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:15 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1066884412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:15.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:15 smithi178 ceph-mon[33233]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:15.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:15 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1066884412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:15.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:15 smithi104 ceph-mon[32726]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:15.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:15 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1066884412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:16.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6834/3785884353,v1:172.21.15.38:6835/3785884353] up:boot 2024-04-25T13:02:16.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: fsmap 3 up:standby 2024-04-25T13:02:16.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:02:16.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:16 smithi038 ceph-mon[29400]: Deploying daemon mds.d on smithi038 2024-04-25T13:02:16.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6834/3785884353,v1:172.21.15.38:6835/3785884353] up:boot 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: fsmap 3 up:standby 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:16 smithi178 ceph-mon[33233]: Deploying daemon mds.d on smithi038 2024-04-25T13:02:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6834/3785884353,v1:172.21.15.38:6835/3785884353] up:boot 2024-04-25T13:02:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: fsmap 3 up:standby 2024-04-25T13:02:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:02:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:16 smithi104 ceph-mon[32726]: Deploying daemon mds.d on smithi038 2024-04-25T13:02:16.855 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 -- 172.21.15.38:0/1720313835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 msgr2=0x7fdae81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/1720313835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae81012b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a410 comp rx=0 tx=0).stop 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 -- 172.21.15.38:0/1720313835 shutdown_connections 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/1720313835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae81029a0 0x7fdae81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/1720313835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 0x7fdae8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/1720313835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae81012b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.978+0000 7fdaef4fc640 1 -- 172.21.15.38:0/1720313835 >> 172.21.15.38:0/1720313835 conn(0x7fdae80fc400 msgr2=0x7fdae80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 -- 172.21.15.38:0/1720313835 shutdown_connections 2024-04-25T13:02:16.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 -- 172.21.15.38:0/1720313835 wait complete. 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 Processor -- start 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 -- start start 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae8069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 0x7fdae819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae81029a0 0x7fdae819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810cba0 con 0x7fdae8100e90 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810ca20 con 0x7fdae81029a0 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.979+0000 7fdaef4fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810cd20 con 0x7fdae8101fe0 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae8069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae8069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32906/0 (socket says 172.21.15.38:32906) 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 -- 172.21.15.38:0/4280522722 learned_addr learned my addr 172.21.15.38:0/4280522722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaeda99640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae81029a0 0x7fdae819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaeca97640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 0x7fdae819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 -- 172.21.15.38:0/4280522722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 msgr2=0x7fdae819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 0x7fdae819d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 -- 172.21.15.38:0/4280522722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae81029a0 msgr2=0x7fdae819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae81029a0 0x7fdae819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 -- 172.21.15.38:0/4280522722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae81a0000 con 0x7fdae8100e90 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaeca97640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 0x7fdae819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaed298640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae8069c50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdae4031570 tx=0x7fdae4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae403f030 con 0x7fdae8100e90 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdae4003cd0 con 0x7fdae8100e90 2024-04-25T13:02:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae81a0230 con 0x7fdae8100e90 2024-04-25T13:02:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4035070 con 0x7fdae8100e90 2024-04-25T13:02:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.980+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae81a07a0 con 0x7fdae8100e90 2024-04-25T13:02:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.982+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae8068d70 con 0x7fdae8100e90 2024-04-25T13:02:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.983+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdae4003800 con 0x7fdae8100e90 2024-04-25T13:02:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.983+0000 7fdade7fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdad40a3bb0 0x7fdad40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:16.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.983+0000 7fdaeca97640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdad40a3bb0 0x7fdad40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:16.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.983+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdae40f0ad0 con 0x7fdae8100e90 2024-04-25T13:02:16.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.984+0000 7fdaeca97640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdad40a3bb0 0x7fdad40a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdad8006fd0 tx=0x7fdad8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:16.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:16.988+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdae4003af0 con 0x7fdae8100e90 2024-04-25T13:02:17.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.245+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdae802d5c0 con 0x7fdae8100e90 2024-04-25T13:02:17.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.245+0000 7fdade7fc640 1 -- 172.21.15.38:0/4280522722 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fdae40bd950 con 0x7fdae8100e90 2024-04-25T13:02:17.244 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:17.244 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:02:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 4 2024-04-25T13:02:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdad40a3bb0 msgr2=0x7fdad40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdad40a3bb0 0x7fdad40a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdad8006fd0 tx=0x7fdad8008040 comp rx=0 tx=0).stop 2024-04-25T13:02:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 msgr2=0x7fdae8069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae8069c50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdae4031570 tx=0x7fdae4004430 comp rx=0 tx=0).stop 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 shutdown_connections 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdad40a3bb0 0x7fdad40a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae81029a0 0x7fdae819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae8101fe0 0x7fdae819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 --2- 172.21.15.38:0/4280522722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100e90 0x7fdae8069c50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 >> 172.21.15.38:0/4280522722 conn(0x7fdae80fc400 msgr2=0x7fdae8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 shutdown_connections 2024-04-25T13:02:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:17.247+0000 7fdaef4fc640 1 -- 172.21.15.38:0/4280522722 wait complete. 2024-04-25T13:02:17.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:17 smithi178 ceph-mon[33233]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:17.537 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:17 smithi038 ceph-mon[29400]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:17.537 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 systemd[1]: Starting Ceph mds.d for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:17.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:17 smithi104 ceph-mon[32726]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:17.821 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 podman[67478]: 2024-04-25 13:02:17.443038996 +0000 UTC m=+0.019202576 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:17.821 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 podman[67478]: 2024-04-25 13:02:17.578354529 +0000 UTC m=+0.154518103 container create 40c4b9555870d8c1a639c6c0e1e763e081896df0d242af0642a6915847016e77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T13:02:18.172 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 podman[67478]: 2024-04-25 13:02:17.824492562 +0000 UTC m=+0.400656147 container init 40c4b9555870d8c1a639c6c0e1e763e081896df0d242af0642a6915847016e77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T13:02:18.173 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 podman[67478]: 2024-04-25 13:02:17.827272389 +0000 UTC m=+0.403435972 container start 40c4b9555870d8c1a639c6c0e1e763e081896df0d242af0642a6915847016e77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:02:18.173 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 ceph-mds[67497]: -- 172.21.15.38:0/2479845627 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555849bcba40 con 0x555849b9ec00 2024-04-25T13:02:18.173 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d[67493]: starting mds.d at 2024-04-25T13:02:18.173 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 bash[67478]: 40c4b9555870d8c1a639c6c0e1e763e081896df0d242af0642a6915847016e77 2024-04-25T13:02:18.173 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:02:17 smithi038 systemd[1]: Started Ceph mds.d for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:18.473 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/4280522722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:18.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:18.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:18.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:18 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/4280522722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:18 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:18.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/4280522722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:18 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:19.258 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:19.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 -- 172.21.15.38:0/1934221084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc1029b0 msgr2=0x7f2bcc1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:19.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 --2- 172.21.15.38:0/1934221084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc1029b0 0x7f2bcc1092d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009ca0 tx=0x7f2bc402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 -- 172.21.15.38:0/1934221084 shutdown_connections 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 --2- 172.21.15.38:0/1934221084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc1029b0 0x7f2bcc1092d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 --2- 172.21.15.38:0/1934221084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 --2- 172.21.15.38:0/1934221084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bcc100ea0 0x7f2bcc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 -- 172.21.15.38:0/1934221084 >> 172.21.15.38:0/1934221084 conn(0x7f2bcc0fc430 msgr2=0x7f2bcc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 -- 172.21.15.38:0/1934221084 shutdown_connections 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 -- 172.21.15.38:0/1934221084 wait complete. 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.377+0000 7f2bd278a640 1 Processor -- start 2024-04-25T13:02:19.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 -- start start 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bcc100ea0 0x7f2bcc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bcc1029b0 0x7f2bcc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10dc50 con 0x7f2bcc101ff0 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10dad0 con 0x7f2bcc1029b0 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd278a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10ddd0 con 0x7f2bcc100ea0 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:19.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32934/0 (socket says 172.21.15.38:32934) 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 -- 172.21.15.38:0/2277595148 learned_addr learned my addr 172.21.15.38:0/2277595148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcbfff640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bcc100ea0 0x7f2bcc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bd0d27640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bcc1029b0 0x7f2bcc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 -- 172.21.15.38:0/2277595148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bcc100ea0 msgr2=0x7f2bcc198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bcc100ea0 0x7f2bcc198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 -- 172.21.15.38:0/2277595148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bcc1029b0 msgr2=0x7f2bcc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bcc1029b0 0x7f2bcc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.378+0000 7f2bcb7fe640 1 -- 172.21.15.38:0/2277595148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc19fe70 con 0x7f2bcc101ff0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.379+0000 7f2bcb7fe640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc198790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400d770 tx=0x7f2bb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.379+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4017070 con 0x7f2bcc101ff0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.379+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc1a0160 con 0x7f2bcc101ff0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.379+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2bb4009c30 con 0x7f2bcc101ff0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.379+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4003d70 con 0x7f2bcc101ff0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.379+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc1a06b0 con 0x7f2bcc101ff0 2024-04-25T13:02:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.380+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bcc1081b0 con 0x7f2bcc101ff0 2024-04-25T13:02:19.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.381+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bb4003f10 con 0x7f2bcc101ff0 2024-04-25T13:02:19.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.382+0000 7f2bc97fa640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2bb80a3bb0 0x7f2bb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:19.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.382+0000 7f2bcbfff640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2bb80a3bb0 0x7f2bb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:19.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.382+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2bb40cfc70 con 0x7f2bcc101ff0 2024-04-25T13:02:19.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.382+0000 7f2bcbfff640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2bb80a3bb0 0x7f2bb80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009bd0 tx=0x7f2bbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:19.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.385+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bb4009de0 con 0x7f2bcc101ff0 2024-04-25T13:02:19.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:19 smithi178 ceph-mon[33233]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:19.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:19 smithi178 ceph-mon[33233]: Deploying daemon mds.e on smithi104 2024-04-25T13:02:19.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:19 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6836/1490443710,v1:172.21.15.38:6837/1490443710] up:boot 2024-04-25T13:02:19.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:19 smithi178 ceph-mon[33233]: fsmap 4 up:standby 2024-04-25T13:02:19.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:19 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:02:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:19 smithi038 ceph-mon[29400]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:19 smithi038 ceph-mon[29400]: Deploying daemon mds.e on smithi104 2024-04-25T13:02:19.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:19 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6836/1490443710,v1:172.21.15.38:6837/1490443710] up:boot 2024-04-25T13:02:19.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:19 smithi038 ceph-mon[29400]: fsmap 4 up:standby 2024-04-25T13:02:19.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:19 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:02:19.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:19 smithi104 ceph-mon[32726]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:19.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:19 smithi104 ceph-mon[32726]: Deploying daemon mds.e on smithi104 2024-04-25T13:02:19.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:19 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6836/1490443710,v1:172.21.15.38:6837/1490443710] up:boot 2024-04-25T13:02:19.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:19 smithi104 ceph-mon[32726]: fsmap 4 up:standby 2024-04-25T13:02:19.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:19 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:02:19.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.613+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bcc02d830 con 0x7f2bcc101ff0 2024-04-25T13:02:19.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.614+0000 7f2bc97fa640 1 -- 172.21.15.38:0/2277595148 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f2bb409c4a0 con 0x7f2bcc101ff0 2024-04-25T13:02:19.613 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:19.613 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:02:19.613 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 5 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2bb80a3bb0 msgr2=0x7f2bb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2bb80a3bb0 0x7f2bb80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009bd0 tx=0x7f2bbc005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 msgr2=0x7f2bcc198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc198790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400d770 tx=0x7f2bb400dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 shutdown_connections 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2bb80a3bb0 0x7f2bb80a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bcc1029b0 0x7f2bcc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bcc101ff0 0x7f2bcc198790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 --2- 172.21.15.38:0/2277595148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bcc100ea0 0x7f2bcc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:19.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 >> 172.21.15.38:0/2277595148 conn(0x7f2bcc0fc430 msgr2=0x7f2bcc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:19.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 shutdown_connections 2024-04-25T13:02:19.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:19.616+0000 7f2bd278a640 1 -- 172.21.15.38:0/2277595148 wait complete. 2024-04-25T13:02:20.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:20 smithi104 ceph-mon[32726]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:20.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:20 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2277595148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:20.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:20 smithi178 ceph-mon[33233]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:20.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:20 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2277595148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:20.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:20 smithi038 ceph-mon[29400]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:20.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:20 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2277595148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:21.073 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:02:20 smithi104 systemd[1]: Started Ceph mds.e for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:21.626 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 -- 172.21.15.38:0/1551628064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8103c00 msgr2=0x7f42d810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 --2- 172.21.15.38:0/1551628064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8103c00 0x7f42d810a490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009cd0 tx=0x7f42d002a8e0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 -- 172.21.15.38:0/1551628064 shutdown_connections 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 --2- 172.21.15.38:0/1551628064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8103c00 0x7f42d810a490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 --2- 172.21.15.38:0/1551628064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42d8103240 0x7f42d81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 --2- 172.21.15.38:0/1551628064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42d8100270 0x7f42d8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.743+0000 7f42dfb87640 1 -- 172.21.15.38:0/1551628064 >> 172.21.15.38:0/1551628064 conn(0x7f42d80fbbc0 msgr2=0x7f42d80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 -- 172.21.15.38:0/1551628064 shutdown_connections 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 -- 172.21.15.38:0/1551628064 wait complete. 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 Processor -- start 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 -- start start 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 0x7f42d81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42d8103240 0x7f42d81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42d8103c00 0x7f42d819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f42d810dc30 con 0x7f42d8100270 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42d810dab0 con 0x7f42d8103c00 2024-04-25T13:02:21.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dfb87640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42d810ddb0 con 0x7f42d8103240 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dd923640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 0x7f42d81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dd923640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 0x7f42d81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32864/0 (socket says 172.21.15.38:32864) 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.744+0000 7f42dd923640 1 -- 172.21.15.38:0/3681570792 learned_addr learned my addr 172.21.15.38:0/3681570792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd122640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42d8103240 0x7f42d81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42de124640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42d8103c00 0x7f42d819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd923640 1 -- 172.21.15.38:0/3681570792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42d8103240 msgr2=0x7f42d81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd923640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42d8103240 0x7f42d81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd923640 1 -- 172.21.15.38:0/3681570792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42d8103c00 msgr2=0x7f42d819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd923640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42d8103c00 0x7f42d819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd923640 1 -- 172.21.15.38:0/3681570792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d819fed0 con 0x7f42d8100270 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dd923640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 0x7f42d81982b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d950 tx=0x7f42d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:21.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4017070 con 0x7f42d8100270 2024-04-25T13:02:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f42d4009c30 con 0x7f42d8100270 2024-04-25T13:02:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d81a01c0 con 0x7f42d8100270 2024-04-25T13:02:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400f620 con 0x7f42d8100270 2024-04-25T13:02:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.745+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81a06c0 con 0x7f42d8100270 2024-04-25T13:02:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.746+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d8109370 con 0x7f42d8100270 2024-04-25T13:02:21.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.747+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f42d401a070 con 0x7f42d8100270 2024-04-25T13:02:21.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.748+0000 7f42ceffd640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f42c40a3bb0 0x7f42c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:21.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.748+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f42d40cfa10 con 0x7f42d8100270 2024-04-25T13:02:21.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.748+0000 7f42dd122640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f42c40a3bb0 0x7f42c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:21.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.749+0000 7f42dd122640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f42c40a3bb0 0x7f42c40a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f42d81997c0 tx=0x7f42c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:21.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.752+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42d4009de0 con 0x7f42d8100270 2024-04-25T13:02:21.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.979+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42d802d5c0 con 0x7f42d8100270 2024-04-25T13:02:21.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.979+0000 7f42ceffd640 1 -- 172.21.15.38:0/3681570792 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f42d409c890 con 0x7f42d8100270 2024-04-25T13:02:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:21.979 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:02:21.979 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 6 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f42c40a3bb0 msgr2=0x7f42c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f42c40a3bb0 0x7f42c40a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f42d81997c0 tx=0x7f42c8005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 msgr2=0x7f42d81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 0x7f42d81982b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d950 tx=0x7f42d400de10 comp rx=0 tx=0).stop 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 shutdown_connections 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f42c40a3bb0 0x7f42c40a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42d8103c00 0x7f42d819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42d8103240 0x7f42d81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 --2- 172.21.15.38:0/3681570792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d8100270 0x7f42d81982b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.981+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 >> 172.21.15.38:0/3681570792 conn(0x7f42d80fbbc0 msgr2=0x7f42d8106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.982+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 shutdown_connections 2024-04-25T13:02:21.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:21.982+0000 7f42dfb87640 1 -- 172.21.15.38:0/3681570792 wait complete. 2024-04-25T13:02:21.990 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:21.990 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:21.990 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: Deploying daemon mds.f on smithi178 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6834/4172911078,v1:172.21.15.104:6835/4172911078] up:boot 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: fsmap 5 up:standby 2024-04-25T13:02:21.991 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:21 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:02:22.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:22.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: Deploying daemon mds.f on smithi178 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6834/4172911078,v1:172.21.15.104:6835/4172911078] up:boot 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: fsmap 5 up:standby 2024-04-25T13:02:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:21 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: Deploying daemon mds.f on smithi178 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6834/4172911078,v1:172.21.15.104:6835/4172911078] up:boot 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: fsmap 5 up:standby 2024-04-25T13:02:22.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:21 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:02:22.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:22 smithi178 ceph-mon[33233]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:22.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:22 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3681570792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:22.989 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:22 smithi178 systemd[1]: Starting Ceph mds.f for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:23.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:22 smithi038 ceph-mon[29400]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:23.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:22 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3681570792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:23.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:22 smithi104 ceph-mon[32726]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:23.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:22 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3681570792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:23.368 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 podman[63679]: 2024-04-25 13:02:23.014417453 +0000 UTC m=+0.018339508 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:23.368 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 podman[63679]: 2024-04-25 13:02:23.134729732 +0000 UTC m=+0.138651772 container create 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:02:23.368 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 podman[63679]: 2024-04-25 13:02:23.370690181 +0000 UTC m=+0.374612221 container init 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T13:02:23.739 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 podman[63679]: 2024-04-25 13:02:23.373196318 +0000 UTC m=+0.377118367 container start 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T13:02:23.739 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 ceph-mds[63698]: -- 172.21.15.178:0/945676123 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4d4af7a40 con 0x55c4d4acac00 2024-04-25T13:02:23.739 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f[63694]: starting mds.f at 2024-04-25T13:02:23.739 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 bash[63679]: 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb 2024-04-25T13:02:23.739 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:02:23 smithi178 systemd[1]: Started Ceph mds.f for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:23.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:24.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 -- 172.21.15.38:0/973844845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101b20 msgr2=0x7fd264101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/973844845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101b20 0x7fd264101fa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009ca0 tx=0x7fd25c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 -- 172.21.15.38:0/973844845 shutdown_connections 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/973844845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101b20 0x7fd264101fa0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/973844845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101180 0x7fd2641015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/973844845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd264100820 0x7fd264100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 -- 172.21.15.38:0/973844845 >> 172.21.15.38:0/973844845 conn(0x7fd2640fc440 msgr2=0x7fd2640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 -- 172.21.15.38:0/973844845 shutdown_connections 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.110+0000 7fd26bfa1640 1 -- 172.21.15.38:0/973844845 wait complete. 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 Processor -- start 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 -- start start 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd264100820 0x7fd26419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 0x7fd26419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101b20 0x7fd2641a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410fe20 con 0x7fd264101180 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410fca0 con 0x7fd264101b20 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26bfa1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410ffa0 con 0x7fd264100820 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26953c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 0x7fd26419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26953c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 0x7fd26419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32884/0 (socket says 172.21.15.38:32884) 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd26953c640 1 -- 172.21.15.38:0/2257709704 learned_addr learned my addr 172.21.15.38:0/2257709704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.111+0000 7fd269d3d640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd264100820 0x7fd26419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26a53e640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101b20 0x7fd2641a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26953c640 1 -- 172.21.15.38:0/2257709704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd264100820 msgr2=0x7fd26419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26953c640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd264100820 0x7fd26419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26953c640 1 -- 172.21.15.38:0/2257709704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101b20 msgr2=0x7fd2641a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26953c640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101b20 0x7fd2641a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26953c640 1 -- 172.21.15.38:0/2257709704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2641a2020 con 0x7fd264101180 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26a53e640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101b20 0x7fd2641a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:24.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26953c640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 0x7fd26419a990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d950 tx=0x7fd25400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:24.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254017070 con 0x7fd264101180 2024-04-25T13:02:24.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd254009c30 con 0x7fd264101180 2024-04-25T13:02:24.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25400f6b0 con 0x7fd264101180 2024-04-25T13:02:24.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2641a2310 con 0x7fd264101180 2024-04-25T13:02:24.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.112+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2641a2860 con 0x7fd264101180 2024-04-25T13:02:24.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.113+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26410b570 con 0x7fd264101180 2024-04-25T13:02:24.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.114+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd2540109c0 con 0x7fd264101180 2024-04-25T13:02:24.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.115+0000 7fd25affd640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd2500a3bb0 0x7fd2500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:24.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.115+0000 7fd269d3d640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd2500a3bb0 0x7fd2500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:24.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.115+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd2540cfce0 con 0x7fd264101180 2024-04-25T13:02:24.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.115+0000 7fd269d3d640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd2500a3bb0 0x7fd2500a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd260005fd0 tx=0x7fd260005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:24.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.119+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd254009de0 con 0x7fd264101180 2024-04-25T13:02:24.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.344+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd264062b50 con 0x7fd264101180 2024-04-25T13:02:24.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.345+0000 7fd25affd640 1 -- 172.21.15.38:0/2257709704 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7fd25409cac0 con 0x7fd264101180 2024-04-25T13:02:24.344 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:24.344 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:02:24.344 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd2500a3bb0 msgr2=0x7fd2500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd2500a3bb0 0x7fd2500a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd260005fd0 tx=0x7fd260005850 comp rx=0 tx=0).stop 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 msgr2=0x7fd26419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 0x7fd26419a990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d950 tx=0x7fd25400de10 comp rx=0 tx=0).stop 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 shutdown_connections 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd2500a3bb0 0x7fd2500a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd264101b20 0x7fd2641a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd264101180 0x7fd26419a990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 --2- 172.21.15.38:0/2257709704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd264100820 0x7fd26419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:24.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 >> 172.21.15.38:0/2257709704 conn(0x7fd2640fc440 msgr2=0x7fd2640fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:24.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 shutdown_connections 2024-04-25T13:02:24.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:24.347+0000 7fd26bfa1640 1 -- 172.21.15.38:0/2257709704 wait complete. 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: Deploying daemon mds.c on smithi178 2024-04-25T13:02:24.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6834/2362992725,v1:172.21.15.178:6835/2362992725] up:boot 2024-04-25T13:02:24.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: fsmap 6 up:standby 2024-04-25T13:02:24.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:02:24.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:24 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2257709704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:24.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:24.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: Deploying daemon mds.c on smithi178 2024-04-25T13:02:24.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6834/2362992725,v1:172.21.15.178:6835/2362992725] up:boot 2024-04-25T13:02:24.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: fsmap 6 up:standby 2024-04-25T13:02:24.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:02:24.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:24 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2257709704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: Deploying daemon mds.c on smithi178 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6834/2362992725,v1:172.21.15.178:6835/2362992725] up:boot 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: fsmap 6 up:standby 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:02:24.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:24 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2257709704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:25.989 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:25 smithi178 systemd[1]: Starting Ceph mds.c for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:26.282 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 podman[63991]: 2024-04-25 13:02:25.904970169 +0000 UTC m=+0.017835451 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:26.283 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 podman[63991]: 2024-04-25 13:02:26.03193707 +0000 UTC m=+0.144802380 container create 13ba5e7f6fdce18eb7cf0c9759d8b7c7a76579575e652e4891ce43a7562acb97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:02:26.283 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 podman[63991]: 2024-04-25 13:02:26.284599414 +0000 UTC m=+0.397464683 container init 13ba5e7f6fdce18eb7cf0c9759d8b7c7a76579575e652e4891ce43a7562acb97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:02:26.357 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.477+0000 7f43bcd4f640 1 -- 172.21.15.38:0/2512157276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102a20 msgr2=0x7f43b8104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.477+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/2512157276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102a20 0x7f43b8104e10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009900 tx=0x7f43ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 -- 172.21.15.38:0/2512157276 shutdown_connections 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/2512157276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8107c80 0x7f43b810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/2512157276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b8105350 0x7f43b8107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/2512157276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102a20 0x7f43b8104e10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 -- 172.21.15.38:0/2512157276 >> 172.21.15.38:0/2512157276 conn(0x7f43b80fc430 msgr2=0x7f43b80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 -- 172.21.15.38:0/2512157276 shutdown_connections 2024-04-25T13:02:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 -- 172.21.15.38:0/2512157276 wait complete. 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 Processor -- start 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 -- start start 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8102a20 0x7f43b8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.478+0000 7f43bcd4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b8105350 0x7f43b81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43bcd4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 0x7f43b819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43bcd4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810dc00 con 0x7f43b8107c80 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43bcd4f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810da80 con 0x7f43b8102a20 2024-04-25T13:02:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43bcd4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810dd80 con 0x7f43b8105350 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 0x7f43b819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 0x7f43b819f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32894/0 (socket says 172.21.15.38:32894) 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 -- 172.21.15.38:0/913467334 learned_addr learned my addr 172.21.15.38:0/913467334 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b5d9b640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b8105350 0x7f43b81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b659c640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8102a20 0x7f43b8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 -- 172.21.15.38:0/913467334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b8105350 msgr2=0x7f43b81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b8105350 0x7f43b81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 -- 172.21.15.38:0/913467334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8102a20 msgr2=0x7f43b8198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8102a20 0x7f43b8198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 -- 172.21.15.38:0/913467334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b819fea0 con 0x7f43b8107c80 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43b6d9d640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 0x7f43b819f830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009fd0 tx=0x7f43b000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0017070 con 0x7f43b8107c80 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b81a0190 con 0x7f43b8107c80 2024-04-25T13:02:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f43b0003e40 con 0x7f43b8107c80 2024-04-25T13:02:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0010460 con 0x7f43b8107c80 2024-04-25T13:02:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.479+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81a06e0 con 0x7f43b8107c80 2024-04-25T13:02:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.480+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b80fdf20 con 0x7f43b8107c80 2024-04-25T13:02:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.482+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43b0004000 con 0x7f43b8107c80 2024-04-25T13:02:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.482+0000 7f43a77fe640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f43a00a3bb0 0x7f43a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:26.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.483+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43b00cfc70 con 0x7f43b8107c80 2024-04-25T13:02:26.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.483+0000 7f43b659c640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f43a00a3bb0 0x7f43a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:26.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.483+0000 7f43b659c640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f43a00a3bb0 0x7f43a00a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43ac000c00 tx=0x7f43ac031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:26.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.489+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43b009caf0 con 0x7f43b8107c80 2024-04-25T13:02:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.718+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43b8062b50 con 0x7f43b8107c80 2024-04-25T13:02:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.718+0000 7f43a77fe640 1 -- 172.21.15.38:0/913467334 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f43b009c3f0 con 0x7f43b8107c80 2024-04-25T13:02:26.716 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:26.717 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:02:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 8 2024-04-25T13:02:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f43a00a3bb0 msgr2=0x7f43a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f43a00a3bb0 0x7f43a00a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43ac000c00 tx=0x7f43ac031510 comp rx=0 tx=0).stop 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 msgr2=0x7f43b819f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 0x7f43b819f830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009fd0 tx=0x7f43b000def0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 shutdown_connections 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f43a00a3bb0 0x7f43a00a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8107c80 0x7f43b819f830 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b8105350 0x7f43b81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 --2- 172.21.15.38:0/913467334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8102a20 0x7f43b8198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 >> 172.21.15.38:0/913467334 conn(0x7f43b80fc430 msgr2=0x7f43b81032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 shutdown_connections 2024-04-25T13:02:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:26.720+0000 7f43bcd4f640 1 -- 172.21.15.38:0/913467334 wait complete. 2024-04-25T13:02:26.728 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:26.728 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.728 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.728 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.728 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:26.729 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:26.729 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:26 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:26.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:26.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 podman[63991]: 2024-04-25 13:02:26.287174259 +0000 UTC m=+0.400039536 container start 13ba5e7f6fdce18eb7cf0c9759d8b7c7a76579575e652e4891ce43a7562acb97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-mds[64011]: -- 172.21.15.178:0/382384510 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639000afa40 con 0x563900082800 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c[64007]: starting mds.c at 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 bash[63991]: 13ba5e7f6fdce18eb7cf0c9759d8b7c7a76579575e652e4891ce43a7562acb97 2024-04-25T13:02:26.739 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:02:26 smithi178 systemd[1]: Started Ceph mds.c for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:26.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:26.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:26.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:26.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:26.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:26 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:27 smithi038 ceph-mon[29400]: Deploying daemon mds.h on smithi104 2024-04-25T13:02:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:27 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] up:boot 2024-04-25T13:02:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:27 smithi038 ceph-mon[29400]: fsmap 7 up:standby 2024-04-25T13:02:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:27 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:02:27.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:27 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/913467334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:27 smithi104 ceph-mon[32726]: Deploying daemon mds.h on smithi104 2024-04-25T13:02:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:27 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] up:boot 2024-04-25T13:02:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:27 smithi104 ceph-mon[32726]: fsmap 7 up:standby 2024-04-25T13:02:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:27 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:02:27.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:27 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/913467334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:27.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:27 smithi178 ceph-mon[33233]: Deploying daemon mds.h on smithi104 2024-04-25T13:02:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:27 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] up:boot 2024-04-25T13:02:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:27 smithi178 ceph-mon[33233]: fsmap 7 up:standby 2024-04-25T13:02:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:27 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:02:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:27 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/913467334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:28.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:28 smithi104 ceph-mon[32726]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:28.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:28.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:28 smithi038 ceph-mon[29400]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:28.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 -- 172.21.15.38:0/1008538321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 msgr2=0x7f4f881012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 --2- 172.21.15.38:0/1008538321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f881012b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009900 tx=0x7f4f7802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 -- 172.21.15.38:0/1008538321 shutdown_connections 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 --2- 172.21.15.38:0/1008538321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f881029a0 0x7f4f881092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 --2- 172.21.15.38:0/1008538321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f88101fe0 0x7f4f88102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 --2- 172.21.15.38:0/1008538321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f881012b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 -- 172.21.15.38:0/1008538321 >> 172.21.15.38:0/1008538321 conn(0x7f4f880fc400 msgr2=0x7f4f880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 -- 172.21.15.38:0/1008538321 shutdown_connections 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.850+0000 7f4f8e220640 1 -- 172.21.15.38:0/1008538321 wait complete. 2024-04-25T13:02:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 Processor -- start 2024-04-25T13:02:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 -- start start 2024-04-25T13:02:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f88198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f88101fe0 0x7f4f88198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f881029a0 0x7f4f8819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810cb50 con 0x7f4f88100e90 2024-04-25T13:02:28.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810c9d0 con 0x7f4f881029a0 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f8e220640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810ccd0 con 0x7f4f88101fe0 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f88198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f88198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32916/0 (socket says 172.21.15.38:32916) 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f877fe640 1 -- 172.21.15.38:0/3165460061 learned_addr learned my addr 172.21.15.38:0/3165460061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f87fff640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f881029a0 0x7f4f8819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.851+0000 7f4f86ffd640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f88101fe0 0x7f4f88198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f877fe640 1 -- 172.21.15.38:0/3165460061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f88101fe0 msgr2=0x7f4f88198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f877fe640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f88101fe0 0x7f4f88198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f877fe640 1 -- 172.21.15.38:0/3165460061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f881029a0 msgr2=0x7f4f8819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f877fe640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f881029a0 0x7f4f8819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f877fe640 1 -- 172.21.15.38:0/3165460061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8819ff30 con 0x7f4f88100e90 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f87fff640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f881029a0 0x7f4f8819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f86ffd640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f88101fe0 0x7f4f88198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:28.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f877fe640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f88198310 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f78005ab0 tx=0x7f4f78032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7803f030 con 0x7f4f88100e90 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f881a01c0 con 0x7f4f88100e90 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4f78003ac0 con 0x7f4f88100e90 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78035070 con 0x7f4f88100e90 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.852+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f881a0700 con 0x7f4f88100e90 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.853+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f881a0a20 con 0x7f4f88100e90 2024-04-25T13:02:28.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.855+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4f78042030 con 0x7f4f88100e90 2024-04-25T13:02:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.856+0000 7f4f84ff9640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4f700a3bb0 0x7f4f700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.856+0000 7f4f86ffd640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4f700a3bb0 0x7f4f700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.856+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4f780f0940 con 0x7f4f88100e90 2024-04-25T13:02:28.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.857+0000 7f4f86ffd640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4f700a3bb0 0x7f4f700a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f88199820 tx=0x7f4f7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:28.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:28.859+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f780bd720 con 0x7f4f88100e90 2024-04-25T13:02:28.966 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:28 smithi104 systemd[1]: Starting Ceph mds.h for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:28.988 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:28 smithi178 ceph-mon[33233]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.085+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f881a0bf0 con 0x7f4f88100e90 2024-04-25T13:02:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.086+0000 7f4f84ff9640 1 -- 172.21.15.38:0/3165460061 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f4f780bd020 con 0x7f4f88100e90 2024-04-25T13:02:29.084 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:29.084 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:02:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 8 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4f700a3bb0 msgr2=0x7f4f700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4f700a3bb0 0x7f4f700a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f88199820 tx=0x7f4f7c005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 msgr2=0x7f4f88198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f88198310 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f78005ab0 tx=0x7f4f78032bc0 comp rx=0 tx=0).stop 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 shutdown_connections 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4f700a3bb0 0x7f4f700a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f881029a0 0x7f4f8819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f88101fe0 0x7f4f88198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 --2- 172.21.15.38:0/3165460061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88100e90 0x7f4f88198310 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 >> 172.21.15.38:0/3165460061 conn(0x7f4f880fc400 msgr2=0x7f4f880690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 shutdown_connections 2024-04-25T13:02:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:29.088+0000 7f4f8e220640 1 -- 172.21.15.38:0/3165460061 wait complete. 2024-04-25T13:02:29.224 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:28 smithi104 podman[62942]: 2024-04-25 13:02:28.871720252 +0000 UTC m=+0.018318673 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:29.224 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:28 smithi104 podman[62942]: 2024-04-25 13:02:28.981830372 +0000 UTC m=+0.128428785 container create aa0d6bbbcaf99649d5c072487eb12815c4411b5e8542c0712371a0d9ae3f030b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:02:29.224 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:29 smithi104 podman[62942]: 2024-04-25 13:02:29.226539259 +0000 UTC m=+0.373137671 container init aa0d6bbbcaf99649d5c072487eb12815c4411b5e8542c0712371a0d9ae3f030b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T13:02:29.543 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:29 smithi104 podman[62942]: 2024-04-25 13:02:29.229206939 +0000 UTC m=+0.375805357 container start aa0d6bbbcaf99649d5c072487eb12815c4411b5e8542c0712371a0d9ae3f030b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T13:02:29.543 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mds[62961]: -- 172.21.15.104:0/2593327667 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55974b6a7a40 con 0x55974b67a800 2024-04-25T13:02:29.543 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h[62957]: starting mds.h at 2024-04-25T13:02:29.543 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:29 smithi104 bash[62942]: aa0d6bbbcaf99649d5c072487eb12815c4411b5e8542c0712371a0d9ae3f030b 2024-04-25T13:02:29.543 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:02:29 smithi104 systemd[1]: Started Ceph mds.h for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3165460061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:29.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:29 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:29.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3165460061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:29.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:29.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:29.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:29 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3165460061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:29 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:30 smithi178 ceph-mon[33233]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:30 smithi178 ceph-mon[33233]: Deploying daemon mds.i on smithi178 2024-04-25T13:02:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:30 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] up:boot 2024-04-25T13:02:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:30 smithi178 ceph-mon[33233]: fsmap 8 up:standby 2024-04-25T13:02:30.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:30 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:02:30.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:30 smithi038 ceph-mon[29400]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:30.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:30 smithi038 ceph-mon[29400]: Deploying daemon mds.i on smithi178 2024-04-25T13:02:30.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:30 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] up:boot 2024-04-25T13:02:30.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:30 smithi038 ceph-mon[29400]: fsmap 8 up:standby 2024-04-25T13:02:30.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:30 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:02:30.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:30 smithi104 ceph-mon[32726]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:30.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:30 smithi104 ceph-mon[32726]: Deploying daemon mds.i on smithi178 2024-04-25T13:02:30.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:30 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] up:boot 2024-04-25T13:02:30.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:30 smithi104 ceph-mon[32726]: fsmap 8 up:standby 2024-04-25T13:02:30.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:30 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:02:31.098 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 -- 172.21.15.38:0/3669701539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470102a30 msgr2=0x7fa470104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 --2- 172.21.15.38:0/3669701539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470102a30 0x7fa470104e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 -- 172.21.15.38:0/3669701539 shutdown_connections 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 --2- 172.21.15.38:0/3669701539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa470107c90 0x7fa47010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 --2- 172.21.15.38:0/3669701539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470105360 0x7fa470107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 --2- 172.21.15.38:0/3669701539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470102a30 0x7fa470104e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 -- 172.21.15.38:0/3669701539 >> 172.21.15.38:0/3669701539 conn(0x7fa4700fc440 msgr2=0x7fa4700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 -- 172.21.15.38:0/3669701539 shutdown_connections 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.216+0000 7fa4773f7640 1 -- 172.21.15.38:0/3669701539 wait complete. 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 Processor -- start 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 -- start start 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa470102a30 0x7fa470198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 0x7fa470198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470107c90 0x7fa47019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010dc60 con 0x7fa470105360 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010dae0 con 0x7fa470102a30 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa4773f7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010dde0 con 0x7fa470107c90 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 0x7fa470198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 0x7fa470198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51126/0 (socket says 172.21.15.38:51126) 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 -- 172.21.15.38:0/910631311 learned_addr learned my addr 172.21.15.38:0/910631311 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa475994640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470107c90 0x7fa47019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa475193640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa470102a30 0x7fa470198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 -- 172.21.15.38:0/910631311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470107c90 msgr2=0x7fa47019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470107c90 0x7fa47019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 -- 172.21.15.38:0/910631311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa470102a30 msgr2=0x7fa470198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.217+0000 7fa474992640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa470102a30 0x7fa470198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa474992640 1 -- 172.21.15.38:0/910631311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47019ff30 con 0x7fa470105360 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa474992640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 0x7fa470198850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa460005fd0 tx=0x7fa46000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460017070 con 0x7fa470105360 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa460009c30 con 0x7fa470105360 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460003de0 con 0x7fa470105360 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4701a0220 con 0x7fa470105360 2024-04-25T13:02:31.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.218+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a0770 con 0x7fa470105360 2024-04-25T13:02:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.219+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4700fdf30 con 0x7fa470105360 2024-04-25T13:02:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.221+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa460003f80 con 0x7fa470105360 2024-04-25T13:02:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.222+0000 7fa4667fc640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.222+0000 7fa475193640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.222+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa4600d0320 con 0x7fa470105360 2024-04-25T13:02:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.222+0000 7fa475193640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa46c031570 tx=0x7fa46c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.225+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa460020030 con 0x7fa470105360 2024-04-25T13:02:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.452+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa47002d5c0 con 0x7fa470105360 2024-04-25T13:02:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.453+0000 7fa4667fc640 1 -- 172.21.15.38:0/910631311 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fa460009de0 con 0x7fa470105360 2024-04-25T13:02:31.452 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:31.452 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T13:02:31.453 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T13:02:31.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa45c0a3bb0 msgr2=0x7fa45c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa46c031570 tx=0x7fa46c005a20 comp rx=0 tx=0).stop 2024-04-25T13:02:31.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 msgr2=0x7fa470198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:31.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 0x7fa470198850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa460005fd0 tx=0x7fa46000dde0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 shutdown_connections 2024-04-25T13:02:31.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470107c90 0x7fa47019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470105360 0x7fa470198850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 --2- 172.21.15.38:0/910631311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa470102a30 0x7fa470198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:31.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 >> 172.21.15.38:0/910631311 conn(0x7fa4700fc440 msgr2=0x7fa470103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:31.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 shutdown_connections 2024-04-25T13:02:31.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:31.455+0000 7fa4773f7640 1 -- 172.21.15.38:0/910631311 wait complete. 2024-04-25T13:02:31.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:31 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/910631311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:31.739 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:31 smithi178 systemd[1]: Starting Ceph mds.i for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:31.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:31 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/910631311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:31.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:31 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/910631311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:32.079 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:31 smithi178 podman[64306]: 2024-04-25 13:02:31.713938333 +0000 UTC m=+0.017889938 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:32.079 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:31 smithi178 podman[64306]: 2024-04-25 13:02:31.828315976 +0000 UTC m=+0.132267578 container create 2a632e5a93d5db05b72a6ce9afb0921f76dacf45c625dac3e0b7fa63f26c4377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T13:02:32.079 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:32 smithi178 podman[64306]: 2024-04-25 13:02:32.055937403 +0000 UTC m=+0.359889005 container init 2a632e5a93d5db05b72a6ce9afb0921f76dacf45c625dac3e0b7fa63f26c4377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T13:02:32.079 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:32 smithi178 podman[64306]: 2024-04-25 13:02:32.058622466 +0000 UTC m=+0.362574067 container start 2a632e5a93d5db05b72a6ce9afb0921f76dacf45c625dac3e0b7fa63f26c4377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T13:02:32.489 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mds[64325]: -- 172.21.15.178:0/2817221111 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a201ceda40 con 0x55a201cc0800 2024-04-25T13:02:32.489 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i[64321]: starting mds.i at 2024-04-25T13:02:32.489 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:32 smithi178 bash[64306]: 2a632e5a93d5db05b72a6ce9afb0921f76dacf45c625dac3e0b7fa63f26c4377 2024-04-25T13:02:32.489 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:02:32 smithi178 systemd[1]: Started Ceph mds.i for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:32.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:32.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:32 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:32.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:32 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:32.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:32 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:33.465 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 -- 172.21.15.38:0/605371978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 msgr2=0x7fdca0104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 --2- 172.21.15.38:0/605371978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca0104ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009900 tx=0x7fdc9002a410 comp rx=0 tx=0).stop 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 -- 172.21.15.38:0/605371978 shutdown_connections 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 --2- 172.21.15.38:0/605371978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 0x7fdca010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 --2- 172.21.15.38:0/605371978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca01053e0 0x7fdca01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 --2- 172.21.15.38:0/605371978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca0104ea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 -- 172.21.15.38:0/605371978 >> 172.21.15.38:0/605371978 conn(0x7fdca00fc400 msgr2=0x7fdca00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 -- 172.21.15.38:0/605371978 shutdown_connections 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.581+0000 7fdca56c2640 1 -- 172.21.15.38:0/605371978 wait complete. 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 Processor -- start 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 -- start start 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca01023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca01053e0 0x7fdca0100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 0x7fdca0100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010dc60 con 0x7fdca0102ab0 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010dae0 con 0x7fdca0107d10 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdca56c2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010dde0 con 0x7fdca01053e0 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdc9effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca01023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdc9effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca01023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51140/0 (socket says 172.21.15.38:51140) 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdc9effd640 1 -- 172.21.15.38:0/2290590380 learned_addr learned my addr 172.21.15.38:0/2290590380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.582+0000 7fdc9f7fe640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 0x7fdca0100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9effd640 1 -- 172.21.15.38:0/2290590380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca01053e0 msgr2=0x7fdca0100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9effd640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca01053e0 0x7fdca0100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9effd640 1 -- 172.21.15.38:0/2290590380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 msgr2=0x7fdca0100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9effd640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 0x7fdca0100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9effd640 1 -- 172.21.15.38:0/2290590380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca0101860 con 0x7fdca0102ab0 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9f7fe640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 0x7fdca0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc9effd640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca01023d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc90005ab0 tx=0x7fdc90032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9003f030 con 0x7fdca0102ab0 2024-04-25T13:02:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca01a23a0 con 0x7fdca0102ab0 2024-04-25T13:02:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdc90003cd0 con 0x7fdca0102ab0 2024-04-25T13:02:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90035070 con 0x7fdca0102ab0 2024-04-25T13:02:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.583+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a28b0 con 0x7fdca0102ab0 2024-04-25T13:02:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.584+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca0068d70 con 0x7fdca0102ab0 2024-04-25T13:02:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.585+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdc90042030 con 0x7fdca0102ab0 2024-04-25T13:02:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.586+0000 7fdc87fff640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc880a3bb0 0x7fdc880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.586+0000 7fdc9e7fc640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc880a3bb0 0x7fdc880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.586+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdc900f02f0 con 0x7fdca0102ab0 2024-04-25T13:02:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.586+0000 7fdc9e7fc640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc880a3bb0 0x7fdc880a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdca0102140 tx=0x7fdc94005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:33.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.590+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc900bd0d0 con 0x7fdca0102ab0 2024-04-25T13:02:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.817+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdca0062b50 con 0x7fdca0102ab0 2024-04-25T13:02:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.818+0000 7fdc87fff640 1 -- 172.21.15.38:0/2290590380 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fdc900bc9d0 con 0x7fdca0102ab0 2024-04-25T13:02:33.816 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:33.816 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:02:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 10 2024-04-25T13:02:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc880a3bb0 msgr2=0x7fdc880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc880a3bb0 0x7fdc880a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdca0102140 tx=0x7fdc94005c70 comp rx=0 tx=0).stop 2024-04-25T13:02:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 msgr2=0x7fdca01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca01023d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc90005ab0 tx=0x7fdc90032d00 comp rx=0 tx=0).stop 2024-04-25T13:02:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 shutdown_connections 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc880a3bb0 0x7fdc880a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0107d10 0x7fdca0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca01053e0 0x7fdca0100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 --2- 172.21.15.38:0/2290590380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca0102ab0 0x7fdca01023d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 >> 172.21.15.38:0/2290590380 conn(0x7fdca00fc400 msgr2=0x7fdca01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 shutdown_connections 2024-04-25T13:02:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:33.820+0000 7fdca56c2640 1 -- 172.21.15.38:0/2290590380 wait complete. 2024-04-25T13:02:33.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:33 smithi038 ceph-mon[29400]: Deploying daemon mds.g on smithi038 2024-04-25T13:02:33.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] up:boot 2024-04-25T13:02:33.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:33 smithi038 ceph-mon[29400]: fsmap 9 up:standby 2024-04-25T13:02:33.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:33 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:02:33.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:33 smithi104 ceph-mon[32726]: Deploying daemon mds.g on smithi038 2024-04-25T13:02:33.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] up:boot 2024-04-25T13:02:33.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:33 smithi104 ceph-mon[32726]: fsmap 9 up:standby 2024-04-25T13:02:33.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:33 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:02:33.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:33 smithi178 ceph-mon[33233]: Deploying daemon mds.g on smithi038 2024-04-25T13:02:33.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] up:boot 2024-04-25T13:02:33.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:33 smithi178 ceph-mon[33233]: fsmap 9 up:standby 2024-04-25T13:02:33.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:33 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:02:34.568 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 systemd[1]: Starting Ceph mds.g for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:34.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:34 smithi038 ceph-mon[29400]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:34.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:34 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2290590380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:34.821 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 podman[68089]: 2024-04-25 13:02:34.532296052 +0000 UTC m=+0.018598292 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:34.821 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 podman[68089]: 2024-04-25 13:02:34.669983059 +0000 UTC m=+0.156285298 container create 3bb9cf690114e204729ed56088d2d4dd9e17347d81c80eebc712d270a730e930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:02:34.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:34 smithi104 ceph-mon[32726]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:34.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:34 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2290590380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:34.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:34 smithi178 ceph-mon[33233]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:34.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:34 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2290590380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:35.321 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 podman[68089]: 2024-04-25 13:02:34.922714199 +0000 UTC m=+0.409016438 container init 3bb9cf690114e204729ed56088d2d4dd9e17347d81c80eebc712d270a730e930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T13:02:35.321 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 podman[68089]: 2024-04-25 13:02:34.925439364 +0000 UTC m=+0.411741611 container start 3bb9cf690114e204729ed56088d2d4dd9e17347d81c80eebc712d270a730e930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T13:02:35.321 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 ceph-mds[68108]: -- 172.21.15.38:0/3521506203 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561328127a40 con 0x5613280fb000 2024-04-25T13:02:35.321 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g[68104]: starting mds.g at 2024-04-25T13:02:35.321 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:35 smithi038 bash[68089]: 3bb9cf690114e204729ed56088d2d4dd9e17347d81c80eebc712d270a730e930 2024-04-25T13:02:35.321 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:02:35 smithi038 systemd[1]: Started Ceph mds.g for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:35.830 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 -- 172.21.15.38:0/3326882073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 msgr2=0x7f8844103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 --2- 172.21.15.38:0/3326882073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844103630 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8838009900 tx=0x7f883802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 -- 172.21.15.38:0/3326882073 shutdown_connections 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 --2- 172.21.15.38:0/3326882073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8844103b70 0x7f884410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 --2- 172.21.15.38:0/3326882073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844103630 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 --2- 172.21.15.38:0/3326882073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8844101fc0 0x7f88441023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 -- 172.21.15.38:0/3326882073 >> 172.21.15.38:0/3326882073 conn(0x7f88440fd530 msgr2=0x7f88440ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 -- 172.21.15.38:0/3326882073 shutdown_connections 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.940+0000 7f884a260640 1 -- 172.21.15.38:0/3326882073 wait complete. 2024-04-25T13:02:35.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 Processor -- start 2024-04-25T13:02:35.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 -- start start 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8844101fc0 0x7f8844075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8844103b70 0x7f8844074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f884410dc20 con 0x7f88441031b0 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f884410daa0 con 0x7f8844101fc0 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f884a260640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f884410dda0 con 0x7f8844103b70 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f8842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f8842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51164/0 (socket says 172.21.15.38:51164) 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f8842ffd640 1 -- 172.21.15.38:0/3546835037 learned_addr learned my addr 172.21.15.38:0/3546835037 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.941+0000 7f88437fe640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8844101fc0 0x7f8844075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8843fff640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8844103b70 0x7f8844074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8842ffd640 1 -- 172.21.15.38:0/3546835037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8844103b70 msgr2=0x7f8844074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8842ffd640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8844103b70 0x7f8844074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8842ffd640 1 -- 172.21.15.38:0/3546835037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8844101fc0 msgr2=0x7f8844075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8842ffd640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8844101fc0 0x7f8844075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:35.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8842ffd640 1 -- 172.21.15.38:0/3546835037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8844075110 con 0x7f88441031b0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8842ffd640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844074340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8838005940 tx=0x7f88380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883803f030 con 0x7f88441031b0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8838003cd0 con 0x7f88441031b0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88441a2380 con 0x7f88441031b0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838035070 con 0x7f88441031b0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.942+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88441a28d0 con 0x7f88441031b0 2024-04-25T13:02:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.943+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8844109400 con 0x7f88441031b0 2024-04-25T13:02:35.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.944+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8838042030 con 0x7f88441031b0 2024-04-25T13:02:35.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.945+0000 7f8840ff9640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f882c0a3bb0 0x7f882c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:35.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.945+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f88380f0820 con 0x7f88441031b0 2024-04-25T13:02:35.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.945+0000 7f88437fe640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f882c0a3bb0 0x7f882c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:35.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.945+0000 7f88437fe640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f882c0a3bb0 0x7f882c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8844103010 tx=0x7f8834005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:35.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:35.949+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88380bd600 con 0x7f88441031b0 2024-04-25T13:02:36.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.177+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f884402d810 con 0x7f88441031b0 2024-04-25T13:02:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.178+0000 7f8840ff9640 1 -- 172.21.15.38:0/3546835037 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f88380bcf00 con 0x7f88441031b0 2024-04-25T13:02:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:36.176 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:02:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.179+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f882c0a3bb0 msgr2=0x7f882c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.179+0000 7f884a260640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f882c0a3bb0 0x7f882c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8844103010 tx=0x7f8834005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 msgr2=0x7f8844074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844074340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8838005940 tx=0x7f88380042f0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 shutdown_connections 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f882c0a3bb0 0x7f882c0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8844103b70 0x7f8844074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88441031b0 0x7f8844074340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 --2- 172.21.15.38:0/3546835037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8844101fc0 0x7f8844075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 >> 172.21.15.38:0/3546835037 conn(0x7f88440fd530 msgr2=0x7f88441065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 shutdown_connections 2024-04-25T13:02:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:36.180+0000 7f884a260640 1 -- 172.21.15.38:0/3546835037 wait complete. 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: Deploying daemon mds.a on smithi038 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6838/3427364907,v1:172.21.15.38:6839/3427364907] up:boot 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: fsmap 10 up:standby 2024-04-25T13:02:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:02:36.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:36.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: Deploying daemon mds.a on smithi038 2024-04-25T13:02:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6838/3427364907,v1:172.21.15.38:6839/3427364907] up:boot 2024-04-25T13:02:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: fsmap 10 up:standby 2024-04-25T13:02:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: Deploying daemon mds.a on smithi038 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6838/3427364907,v1:172.21.15.38:6839/3427364907] up:boot 2024-04-25T13:02:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: fsmap 10 up:standby 2024-04-25T13:02:36.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:02:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:37 smithi038 ceph-mon[29400]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:37 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3546835037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:37.322 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 systemd[1]: Starting Ceph mds.a for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:37 smithi178 ceph-mon[33233]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:37 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3546835037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:37 smithi104 ceph-mon[32726]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:37 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3546835037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:37.701 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 podman[68440]: 2024-04-25 13:02:37.353862139 +0000 UTC m=+0.018857919 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:37.702 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 podman[68440]: 2024-04-25 13:02:37.476029883 +0000 UTC m=+0.141025652 container create a7e2e7b9f3baf1c32d754c315a48088ec45066c03e8f29f2496c0418cd169a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:02:37.702 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 podman[68440]: 2024-04-25 13:02:37.70363317 +0000 UTC m=+0.368628938 container init a7e2e7b9f3baf1c32d754c315a48088ec45066c03e8f29f2496c0418cd169a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T13:02:38.071 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 podman[68440]: 2024-04-25 13:02:37.706374717 +0000 UTC m=+0.371370487 container start a7e2e7b9f3baf1c32d754c315a48088ec45066c03e8f29f2496c0418cd169a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:02:38.071 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 ceph-mds[68459]: -- 172.21.15.38:0/1057710532 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56131c7e5a40 con 0x56131c7b8800 2024-04-25T13:02:38.071 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a[68455]: starting mds.a at 2024-04-25T13:02:38.071 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 bash[68440]: a7e2e7b9f3baf1c32d754c315a48088ec45066c03e8f29f2496c0418cd169a27 2024-04-25T13:02:38.071 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:02:37 smithi038 systemd[1]: Started Ceph mds.a for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:38.190 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:38.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/2082481411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 msgr2=0x7ff0e410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/2082481411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e410a0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009ca0 tx=0x7ff0dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/2082481411 shutdown_connections 2024-04-25T13:02:38.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/2082481411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e410a0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/2082481411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0e41053a0 0x7ff0e4107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/2082481411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e4102a70 0x7ff0e4104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/2082481411 >> 172.21.15.38:0/2082481411 conn(0x7ff0e40fc400 msgr2=0x7ff0e40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/2082481411 shutdown_connections 2024-04-25T13:02:38.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.300+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/2082481411 wait complete. 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 Processor -- start 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 -- start start 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e4102a70 0x7ff0e41023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0e41053a0 0x7ff0e4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e4100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e410dc00 con 0x7ff0e4107cd0 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e410da80 con 0x7ff0e4102a70 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ec2e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e410dd80 con 0x7ff0e41053a0 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e4100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea07d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e4102a70 0x7ff0e41023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e4100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51208/0 (socket says 172.21.15.38:51208) 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 -- 172.21.15.38:0/503767798 learned_addr learned my addr 172.21.15.38:0/503767798 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0e987c640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0e41053a0 0x7ff0e4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 -- 172.21.15.38:0/503767798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0e41053a0 msgr2=0x7ff0e4100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0e41053a0 0x7ff0e4100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 -- 172.21.15.38:0/503767798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e4102a70 msgr2=0x7ff0e41023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e4102a70 0x7ff0e41023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 -- 172.21.15.38:0/503767798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e41017f0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.301+0000 7ff0ea87e640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e4100f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009dc0 tx=0x7ff0dc02f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.302+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc003cf0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.302+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0dc004510 con 0x7ff0e4107cd0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.302+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e41a2360 con 0x7ff0e4107cd0 2024-04-25T13:02:38.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.302+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc02fdb0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.302+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e41a28b0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.303+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e40fdef0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.304+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff0dc0046c0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.304+0000 7ff0db7fe640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0d00a3bb0 0x7ff0d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:38.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.305+0000 7ff0ea07d640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0d00a3bb0 0x7ff0d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:38.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.305+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff0dc0f18a0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.305+0000 7ff0ea07d640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0d00a3bb0 0x7ff0d00a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0005d00 tx=0x7ff0e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:38.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.308+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0dc0be680 con 0x7ff0e4107cd0 2024-04-25T13:02:38.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.535+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0e402d5c0 con 0x7ff0e4107cd0 2024-04-25T13:02:38.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.536+0000 7ff0db7fe640 1 -- 172.21.15.38:0/503767798 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7ff0dc0bdf80 con 0x7ff0e4107cd0 2024-04-25T13:02:38.534 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:38.534 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:02:38.534 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 12 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0d00a3bb0 msgr2=0x7ff0d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0d00a3bb0 0x7ff0d00a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0005d00 tx=0x7ff0e000b410 comp rx=0 tx=0).stop 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 msgr2=0x7ff0e4100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e4100f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009dc0 tx=0x7ff0dc02f460 comp rx=0 tx=0).stop 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 shutdown_connections 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0d00a3bb0 0x7ff0d00a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e4107cd0 0x7ff0e4100f60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0e41053a0 0x7ff0e4100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 --2- 172.21.15.38:0/503767798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e4102a70 0x7ff0e41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 >> 172.21.15.38:0/503767798 conn(0x7ff0e40fc400 msgr2=0x7ff0e41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:38.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 shutdown_connections 2024-04-25T13:02:38.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:38.538+0000 7ff0ec2e1640 1 -- 172.21.15.38:0/503767798 wait complete. 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: Deploying daemon mds.b on smithi104 2024-04-25T13:02:39.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] up:boot 2024-04-25T13:02:39.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: fsmap 11 up:standby 2024-04-25T13:02:39.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:02:39.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:38 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/503767798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:39.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:39.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: Deploying daemon mds.b on smithi104 2024-04-25T13:02:39.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] up:boot 2024-04-25T13:02:39.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: fsmap 11 up:standby 2024-04-25T13:02:39.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:02:39.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:38 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/503767798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: Deploying daemon mds.b on smithi104 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] up:boot 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: fsmap 11 up:standby 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:02:39.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:38 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/503767798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:40.311 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 systemd[1]: Starting Ceph mds.b for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:02:40.548 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:40.573 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 podman[63254]: 2024-04-25 13:02:40.215916088 +0000 UTC m=+0.018213058 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:02:40.573 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 podman[63254]: 2024-04-25 13:02:40.345142651 +0000 UTC m=+0.147439652 container create 8270cd36762eb1769a3cdf3653fdd988b48c595fe18bdb66538755e08a42b31f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 -- 172.21.15.38:0/209429152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f84103b70 msgr2=0x7f8f8410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/209429152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f84103b70 0x7f8f8410a400 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009ca0 tx=0x7f8f8002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 -- 172.21.15.38:0/209429152 shutdown_connections 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/209429152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f84103b70 0x7f8f8410a400 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/209429152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f841031b0 0x7f8f84103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/209429152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f84101fc0 0x7f8f841023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 -- 172.21.15.38:0/209429152 >> 172.21.15.38:0/209429152 conn(0x7f8f840fd550 msgr2=0x7f8f840ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 -- 172.21.15.38:0/209429152 shutdown_connections 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.665+0000 7f8f8bff7640 1 -- 172.21.15.38:0/209429152 wait complete. 2024-04-25T13:02:40.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 Processor -- start 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 -- start start 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f84101fc0 0x7f8f84198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 0x7f8f84198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f84103b70 0x7f8f8419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8410dc20 con 0x7f8f841031b0 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8410daa0 con 0x7f8f84103b70 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8bff7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8410dda0 con 0x7f8f84101fc0 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f89592640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 0x7f8f84198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f89592640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 0x7f8f84198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51132/0 (socket says 172.21.15.38:51132) 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f89592640 1 -- 172.21.15.38:0/2257270317 learned_addr learned my addr 172.21.15.38:0/2257270317 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.666+0000 7f8f8a594640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f84103b70 0x7f8f8419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89d93640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f84101fc0 0x7f8f84198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89592640 1 -- 172.21.15.38:0/2257270317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f84101fc0 msgr2=0x7f8f84198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89592640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f84101fc0 0x7f8f84198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89592640 1 -- 172.21.15.38:0/2257270317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f84103b70 msgr2=0x7f8f8419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89592640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f84103b70 0x7f8f8419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89592640 1 -- 172.21.15.38:0/2257270317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8419fe60 con 0x7f8f841031b0 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f89592640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 0x7f8f84198780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400e9f0 tx=0x7f8f7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400cd80 con 0x7f8f841031b0 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f841a0150 con 0x7f8f841031b0 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8f74010900 con 0x7f8f841031b0 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74018920 con 0x7f8f841031b0 2024-04-25T13:02:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.667+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f841a06a0 con 0x7f8f841031b0 2024-04-25T13:02:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.668+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f84109280 con 0x7f8f841031b0 2024-04-25T13:02:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.669+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f74010ab0 con 0x7f8f841031b0 2024-04-25T13:02:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.670+0000 7f8f7affd640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8f700a3bb0 0x7f8f700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.670+0000 7f8f89d93640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8f700a3bb0 0x7f8f700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.670+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8f740d0800 con 0x7f8f841031b0 2024-04-25T13:02:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.670+0000 7f8f89d93640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8f700a3bb0 0x7f8f700a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c005fd0 tx=0x7f8f7c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:40.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.674+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f7409d680 con 0x7f8f841031b0 2024-04-25T13:02:40.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.920+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f8402d5c0 con 0x7f8f841031b0 2024-04-25T13:02:40.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.921+0000 7f8f7affd640 1 -- 172.21.15.38:0/2257270317 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f8f7409cf80 con 0x7f8f841031b0 2024-04-25T13:02:40.919 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:40.920 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T13:02:40.920 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-04-25T13:02:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8f700a3bb0 msgr2=0x7f8f700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8f700a3bb0 0x7f8f700a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c005fd0 tx=0x7f8f7c005850 comp rx=0 tx=0).stop 2024-04-25T13:02:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 msgr2=0x7f8f84198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 0x7f8f84198780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400e9f0 tx=0x7f8f7400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 shutdown_connections 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8f700a3bb0 0x7f8f700a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f84103b70 0x7f8f8419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f841031b0 0x7f8f84198780 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 --2- 172.21.15.38:0/2257270317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f84101fc0 0x7f8f84198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 >> 172.21.15.38:0/2257270317 conn(0x7f8f840fd550 msgr2=0x7f8f84106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 shutdown_connections 2024-04-25T13:02:40.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:40.923+0000 7f8f8bff7640 1 -- 172.21.15.38:0/2257270317 wait complete. 2024-04-25T13:02:40.932 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:41.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.037+0000 7fed2bca7640 1 -- 172.21.15.38:0/4075461524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24100e90 msgr2=0x7fed241012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.037+0000 7fed2bca7640 1 --2- 172.21.15.38:0/4075461524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24100e90 0x7fed241012b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed20009900 tx=0x7fed2002a410 comp rx=0 tx=0).stop 2024-04-25T13:02:41.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.037+0000 7fed2bca7640 1 -- 172.21.15.38:0/4075461524 shutdown_connections 2024-04-25T13:02:41.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 --2- 172.21.15.38:0/4075461524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed241029a0 0x7fed241092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 --2- 172.21.15.38:0/4075461524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24101fe0 0x7fed24102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 --2- 172.21.15.38:0/4075461524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24100e90 0x7fed241012b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- 172.21.15.38:0/4075461524 >> 172.21.15.38:0/4075461524 conn(0x7fed240fc400 msgr2=0x7fed240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- 172.21.15.38:0/4075461524 shutdown_connections 2024-04-25T13:02:41.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- 172.21.15.38:0/4075461524 wait complete. 2024-04-25T13:02:41.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 Processor -- start 2024-04-25T13:02:41.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- start start 2024-04-25T13:02:41.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24100e90 0x7fed24069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 0x7fed2419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed241029a0 0x7fed2419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed2410cb20 con 0x7fed24101fe0 2024-04-25T13:02:41.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed2410c9a0 con 0x7fed24100e90 2024-04-25T13:02:41.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed2bca7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed2410cca0 con 0x7fed241029a0 2024-04-25T13:02:41.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.038+0000 7fed29242640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 0x7fed2419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29242640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 0x7fed2419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51160/0 (socket says 172.21.15.38:51160) 2024-04-25T13:02:41.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29242640 1 -- 172.21.15.38:0/1291499482 learned_addr learned my addr 172.21.15.38:0/1291499482 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:41.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24100e90 0x7fed24069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed2a244640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed241029a0 0x7fed2419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 -- 172.21.15.38:0/1291499482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed241029a0 msgr2=0x7fed2419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed241029a0 0x7fed2419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 -- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 msgr2=0x7fed2419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 0x7fed2419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 -- 172.21.15.38:0/1291499482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2419fee0 con 0x7fed24100e90 2024-04-25T13:02:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29242640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 0x7fed2419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed29a43640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24100e90 0x7fed24069cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed20006010 tx=0x7fed200042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2003f030 con 0x7fed24100e90 2024-04-25T13:02:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed241a0170 con 0x7fed24100e90 2024-04-25T13:02:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241a06b0 con 0x7fed24100e90 2024-04-25T13:02:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fed20003cd0 con 0x7fed24100e90 2024-04-25T13:02:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.039+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20035070 con 0x7fed24100e90 2024-04-25T13:02:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.040+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed241081a0 con 0x7fed24100e90 2024-04-25T13:02:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.042+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed20042030 con 0x7fed24100e90 2024-04-25T13:02:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.042+0000 7fed1affd640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fed100a3bb0 0x7fed100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.042+0000 7fed29242640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fed100a3bb0 0x7fed100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.042+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fed200f0650 con 0x7fed24100e90 2024-04-25T13:02:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.043+0000 7fed29242640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fed100a3bb0 0x7fed100a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed24069a20 tx=0x7fed14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.051+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed200bcde0 con 0x7fed24100e90 2024-04-25T13:02:41.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mon[32726]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:41.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.074 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 podman[63254]: 2024-04-25 13:02:40.614856018 +0000 UTC m=+0.417152973 container init 8270cd36762eb1769a3cdf3653fdd988b48c595fe18bdb66538755e08a42b31f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:02:41.074 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 podman[63254]: 2024-04-25 13:02:40.617909184 +0000 UTC m=+0.420206140 container start 8270cd36762eb1769a3cdf3653fdd988b48c595fe18bdb66538755e08a42b31f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:02:41.075 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-mds[63273]: -- 172.21.15.104:0/1984052934 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4c785da40 con 0x55e4c7831000 2024-04-25T13:02:41.075 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: starting mds.b at 2024-04-25T13:02:41.075 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 bash[63254]: 8270cd36762eb1769a3cdf3653fdd988b48c595fe18bdb66538755e08a42b31f 2024-04-25T13:02:41.075 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:40 smithi104 systemd[1]: Started Ceph mds.b for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:02:41.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.238+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed24062b50 con 0x7fed24100e90 2024-04-25T13:02:41.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.240+0000 7fed1affd640 1 -- 172.21.15.38:0/1291499482 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18409 (secure 0 0 0) 0x7fed200c10e0 con 0x7fed24100e90 2024-04-25T13:02:41.239 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:41.239 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:01:56.666084+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:41.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:40 smithi178 ceph-mon[33233]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:41.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:40 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fed100a3bb0 msgr2=0x7fed100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fed100a3bb0 0x7fed100a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed24069a20 tx=0x7fed14005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24100e90 msgr2=0x7fed24069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24100e90 0x7fed24069cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed20006010 tx=0x7fed200042f0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 shutdown_connections 2024-04-25T13:02:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fed100a3bb0 0x7fed100a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed241029a0 0x7fed2419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed24101fe0 0x7fed2419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 --2- 172.21.15.38:0/1291499482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed24100e90 0x7fed24069cb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 >> 172.21.15.38:0/1291499482 conn(0x7fed240fc400 msgr2=0x7fed24105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 shutdown_connections 2024-04-25T13:02:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.242+0000 7fed2bca7640 1 -- 172.21.15.38:0/1291499482 wait complete. 2024-04-25T13:02:41.250 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:40 smithi038 ceph-mon[29400]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:41.251 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.251 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.251 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.251 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.251 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:40 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:41.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:02:41.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:02:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 -- 172.21.15.38:0/3627268384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c100e90 msgr2=0x7fd64c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 --2- 172.21.15.38:0/3627268384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c100e90 0x7fd64c1012b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd640009900 tx=0x7fd64002a410 comp rx=0 tx=0).stop 2024-04-25T13:02:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 -- 172.21.15.38:0/3627268384 shutdown_connections 2024-04-25T13:02:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 --2- 172.21.15.38:0/3627268384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd64c1029a0 0x7fd64c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 --2- 172.21.15.38:0/3627268384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c101fe0 0x7fd64c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 --2- 172.21.15.38:0/3627268384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c100e90 0x7fd64c1012b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 -- 172.21.15.38:0/3627268384 >> 172.21.15.38:0/3627268384 conn(0x7fd64c0fc400 msgr2=0x7fd64c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 -- 172.21.15.38:0/3627268384 shutdown_connections 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.362+0000 7fd653c05640 1 -- 172.21.15.38:0/3627268384 wait complete. 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 Processor -- start 2024-04-25T13:02:41.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 -- start start 2024-04-25T13:02:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd64c100e90 0x7fd64c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 0x7fd64c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c1029a0 0x7fd64c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c10cb00 con 0x7fd64c101fe0 2024-04-25T13:02:41.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c10c980 con 0x7fd64c1029a0 2024-04-25T13:02:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd653c05640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c10cc80 con 0x7fd64c100e90 2024-04-25T13:02:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 0x7fd64c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6519a1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd64c100e90 0x7fd64c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 0x7fd64c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51180/0 (socket says 172.21.15.38:51180) 2024-04-25T13:02:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 -- 172.21.15.38:0/1960467125 learned_addr learned my addr 172.21.15.38:0/1960467125 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:41.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6521a2640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c1029a0 0x7fd64c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 -- 172.21.15.38:0/1960467125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd64c100e90 msgr2=0x7fd64c069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd64c100e90 0x7fd64c069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 -- 172.21.15.38:0/1960467125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c1029a0 msgr2=0x7fd64c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c1029a0 0x7fd64c1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 -- 172.21.15.38:0/1960467125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c1a2160 con 0x7fd64c101fe0 2024-04-25T13:02:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.363+0000 7fd6511a0640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 0x7fd64c19f420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e9c0 tx=0x7fd64800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.364+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64800cd80 con 0x7fd64c101fe0 2024-04-25T13:02:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.364+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd6480035f0 con 0x7fd64c101fe0 2024-04-25T13:02:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.364+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64c1a23f0 con 0x7fd64c101fe0 2024-04-25T13:02:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.364+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648010690 con 0x7fd64c101fe0 2024-04-25T13:02:41.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.364+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64c1a2940 con 0x7fd64c101fe0 2024-04-25T13:02:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.365+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64c1082c0 con 0x7fd64c101fe0 2024-04-25T13:02:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.366+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd6480037a0 con 0x7fd64c101fe0 2024-04-25T13:02:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.367+0000 7fd63effd640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd6380a3be0 0x7fd6380a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.367+0000 7fd6519a1640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd6380a3be0 0x7fd6380a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.367+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd6480d0310 con 0x7fd64c101fe0 2024-04-25T13:02:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.367+0000 7fd6519a1640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd6380a3be0 0x7fd6380a60a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd640000c00 tx=0x7fd640031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:41.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.372+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd64809d190 con 0x7fd64c101fe0 2024-04-25T13:02:41.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.594+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd64c02d810 con 0x7fd64c101fe0 2024-04-25T13:02:41.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.595+0000 7fd63effd640 1 -- 172.21.15.38:0/1960467125 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd64809ca90 con 0x7fd64c101fe0 2024-04-25T13:02:41.593 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:02:41.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd6380a3be0 msgr2=0x7fd6380a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd6380a3be0 0x7fd6380a60a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd640000c00 tx=0x7fd640031510 comp rx=0 tx=0).stop 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 msgr2=0x7fd64c19f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 0x7fd64c19f420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e9c0 tx=0x7fd64800ee80 comp rx=0 tx=0).stop 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 shutdown_connections 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd6380a3be0 0x7fd6380a60a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c1029a0 0x7fd64c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd64c101fe0 0x7fd64c19f420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 --2- 172.21.15.38:0/1960467125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd64c100e90 0x7fd64c069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 >> 172.21.15.38:0/1960467125 conn(0x7fd64c0fc400 msgr2=0x7fd64c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 shutdown_connections 2024-04-25T13:02:41.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.597+0000 7fd653c05640 1 -- 172.21.15.38:0/1960467125 wait complete. 2024-04-25T13:02:41.608 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:02:41.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 -- 172.21.15.38:0/1731600184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468101ff0 msgr2=0x7f6468102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 --2- 172.21.15.38:0/1731600184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468101ff0 0x7f6468102470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f645c009900 tx=0x7f645c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 -- 172.21.15.38:0/1731600184 shutdown_connections 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 --2- 172.21.15.38:0/1731600184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64681029b0 0x7f64681092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 --2- 172.21.15.38:0/1731600184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468101ff0 0x7f6468102470 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 --2- 172.21.15.38:0/1731600184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468100ea0 0x7f64681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 -- 172.21.15.38:0/1731600184 >> 172.21.15.38:0/1731600184 conn(0x7f64680fc430 msgr2=0x7f64680fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 -- 172.21.15.38:0/1731600184 shutdown_connections 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.712+0000 7f646ccbb640 1 -- 172.21.15.38:0/1731600184 wait complete. 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 Processor -- start 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 -- start start 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 0x7f64681982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468101ff0 0x7f64681987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64681029b0 0x7f646819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f646810cb40 con 0x7f6468100ea0 2024-04-25T13:02:41.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f646810c9c0 con 0x7f64681029b0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646ccbb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f646810ccc0 con 0x7f6468101ff0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 0x7f64681982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 0x7f64681982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51198/0 (socket says 172.21.15.38:51198) 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 -- 172.21.15.38:0/881900434 learned_addr learned my addr 172.21.15.38:0/881900434 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f6465d9b640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468101ff0 0x7f64681987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f6466d9d640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64681029b0 0x7f646819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 -- 172.21.15.38:0/881900434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468101ff0 msgr2=0x7f64681987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468101ff0 0x7f64681987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 -- 172.21.15.38:0/881900434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64681029b0 msgr2=0x7f646819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64681029b0 0x7f646819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 -- 172.21.15.38:0/881900434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646819fec0 con 0x7f6468100ea0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f6465d9b640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468101ff0 0x7f64681987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.713+0000 7f646659c640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 0x7f64681982a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f645800c970 tx=0x7f645800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.714+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458007bb0 con 0x7f6468100ea0 2024-04-25T13:02:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.714+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6458007d50 con 0x7f6468100ea0 2024-04-25T13:02:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.714+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64681a01b0 con 0x7f6468100ea0 2024-04-25T13:02:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.714+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645800f450 con 0x7f6468100ea0 2024-04-25T13:02:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.714+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64681a0680 con 0x7f6468100ea0 2024-04-25T13:02:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.715+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64681a0940 con 0x7f6468100ea0 2024-04-25T13:02:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.716+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f64580046c0 con 0x7f6468100ea0 2024-04-25T13:02:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.716+0000 7f64577fe640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.716+0000 7f6465d9b640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.716+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f64580cfdc0 con 0x7f6468100ea0 2024-04-25T13:02:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.717+0000 7f6465d9b640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f645c000c00 tx=0x7f645c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:41.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.721+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6458014030 con 0x7f6468100ea0 2024-04-25T13:02:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.911+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f64681a0b10 con 0x7f6468100ea0 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.911+0000 7f64577fe640 1 -- 172.21.15.38:0/881900434 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f645809cc40 con 0x7f6468100ea0 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:02:41.910 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 msgr2=0x7f64500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f645c000c00 tx=0x7f645c031510 comp rx=0 tx=0).stop 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 msgr2=0x7f64681982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 0x7f64681982a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f645800c970 tx=0x7f645800ce30 comp rx=0 tx=0).stop 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 shutdown_connections 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64681029b0 0x7f646819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6468101ff0 0x7f64681987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 --2- 172.21.15.38:0/881900434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6468100ea0 0x7f64681982a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 >> 172.21.15.38:0/881900434 conn(0x7f64680fc430 msgr2=0x7f64680fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 shutdown_connections 2024-04-25T13:02:41.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:41.914+0000 7f646ccbb640 1 -- 172.21.15.38:0/881900434 wait complete. 2024-04-25T13:02:41.924 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:02:41.924 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] up:boot 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: fsmap 12 up:standby 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2257270317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1291499482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1960467125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 -- 172.21.15.38:0/99977989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464103190 msgr2=0x7f9464103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 --2- 172.21.15.38:0/99977989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464103190 0x7f9464103610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9458009900 tx=0x7f945802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 -- 172.21.15.38:0/99977989 shutdown_connections 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 --2- 172.21.15.38:0/99977989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9464103b50 0x7f946410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 --2- 172.21.15.38:0/99977989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464103190 0x7f9464103610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 --2- 172.21.15.38:0/99977989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9464101fa0 0x7f94641023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 -- 172.21.15.38:0/99977989 >> 172.21.15.38:0/99977989 conn(0x7f94640fd530 msgr2=0x7f94640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 -- 172.21.15.38:0/99977989 shutdown_connections 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.035+0000 7f946a660640 1 -- 172.21.15.38:0/99977989 wait complete. 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 Processor -- start 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 -- start start 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 0x7f94641982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9464103190 0x7f94641987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9464103b50 0x7f946419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f946410dc00 con 0x7f9464101fa0 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f946410da80 con 0x7f9464103b50 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f946a660640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f946410dd80 con 0x7f9464103190 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 0x7f94641982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 0x7f94641982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51206/0 (socket says 172.21.15.38:51206) 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 -- 172.21.15.38:0/3796509073 learned_addr learned my addr 172.21.15.38:0/3796509073 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9468bfd640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9464103b50 0x7f946419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f94637fe640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9464103190 0x7f94641987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 -- 172.21.15.38:0/3796509073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9464103190 msgr2=0x7f94641987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9464103190 0x7f94641987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 -- 172.21.15.38:0/3796509073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9464103b50 msgr2=0x7f946419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:42.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9464103b50 0x7f946419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.036+0000 7f9463fff640 1 -- 172.21.15.38:0/3796509073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946419fed0 con 0x7f9464101fa0 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.037+0000 7f9463fff640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 0x7f94641982b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f945400e9c0 tx=0x7f945400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.037+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945400cd80 con 0x7f9464101fa0 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.037+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94540035f0 con 0x7f9464101fa0 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.037+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94641a01c0 con 0x7f9464101fa0 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.037+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454010690 con 0x7f9464101fa0 2024-04-25T13:02:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.037+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94641a06e0 con 0x7f9464101fa0 2024-04-25T13:02:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.038+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464109390 con 0x7f9464101fa0 2024-04-25T13:02:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.039+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f94540037a0 con 0x7f9464101fa0 2024-04-25T13:02:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.040+0000 7f94617fa640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f944c0a3bb0 0x7f944c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.040+0000 7f94637fe640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f944c0a3bb0 0x7f944c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.040+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f94540d0410 con 0x7f9464101fa0 2024-04-25T13:02:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.040+0000 7f94637fe640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f944c0a3bb0 0x7f944c0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f9458031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:42.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.043+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f945409d320 con 0x7f9464101fa0 2024-04-25T13:02:42.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.231+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f946402d5c0 con 0x7f9464101fa0 2024-04-25T13:02:42.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] up:boot 2024-04-25T13:02:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: fsmap 12 up:standby 2024-04-25T13:02:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:02:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2257270317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1291499482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1960467125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] up:boot 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: fsmap 12 up:standby 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2257270317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1291499482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:42.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1960467125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:02:42.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.878+0000 7f94617fa640 1 -- 172.21.15.38:0/3796509073 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f945409cc20 con 0x7f9464101fa0 2024-04-25T13:02:42.877 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T13:02:42.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f944c0a3bb0 msgr2=0x7f944c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f944c0a3bb0 0x7f944c0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f9458031510 comp rx=0 tx=0).stop 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 msgr2=0x7f94641982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 0x7f94641982b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f945400e9c0 tx=0x7f945400ee80 comp rx=0 tx=0).stop 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 shutdown_connections 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f944c0a3bb0 0x7f944c0a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9464103b50 0x7f946419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9464103190 0x7f94641987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 --2- 172.21.15.38:0/3796509073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9464101fa0 0x7f94641982b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.881+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 >> 172.21.15.38:0/3796509073 conn(0x7f94640fd530 msgr2=0x7f9464106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.882+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 shutdown_connections 2024-04-25T13:02:42.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:42.882+0000 7f946a660640 1 -- 172.21.15.38:0/3796509073 wait complete. 2024-04-25T13:02:42.893 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:02:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 -- 172.21.15.38:0/2222233275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c105360 msgr2=0x7ff33c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 --2- 172.21.15.38:0/2222233275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c105360 0x7ff33c107750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009900 tx=0x7ff32c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 -- 172.21.15.38:0/2222233275 shutdown_connections 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 --2- 172.21.15.38:0/2222233275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c107c90 0x7ff33c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 --2- 172.21.15.38:0/2222233275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c105360 0x7ff33c107750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 --2- 172.21.15.38:0/2222233275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff33c102a30 0x7ff33c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 -- 172.21.15.38:0/2222233275 >> 172.21.15.38:0/2222233275 conn(0x7ff33c0fc440 msgr2=0x7ff33c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 -- 172.21.15.38:0/2222233275 shutdown_connections 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.005+0000 7ff344530640 1 -- 172.21.15.38:0/2222233275 wait complete. 2024-04-25T13:02:43.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 Processor -- start 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 -- start start 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 0x7ff33c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c105360 0x7ff33c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff33c107c90 0x7ff33c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c10dc10 con 0x7ff33c102a30 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c10da90 con 0x7ff33c105360 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff344530640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c10dd90 con 0x7ff33c107c90 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff3422cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 0x7ff33c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff3422cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 0x7ff33c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51224/0 (socket says 172.21.15.38:51224) 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff3422cc640 1 -- 172.21.15.38:0/2513467201 learned_addr learned my addr 172.21.15.38:0/2513467201 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff341acb640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c105360 0x7ff33c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.006+0000 7ff342acd640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff33c107c90 0x7ff33c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3422cc640 1 -- 172.21.15.38:0/2513467201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff33c107c90 msgr2=0x7ff33c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3422cc640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff33c107c90 0x7ff33c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3422cc640 1 -- 172.21.15.38:0/2513467201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c105360 msgr2=0x7ff33c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3422cc640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c105360 0x7ff33c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3422cc640 1 -- 172.21.15.38:0/2513467201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c19ff80 con 0x7ff33c102a30 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff341acb640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c105360 0x7ff33c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3422cc640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 0x7ff33c1983b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff33800e9f0 tx=0x7ff33800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33800ccc0 con 0x7ff33c102a30 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c1a0270 con 0x7ff33c102a30 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff33800ce60 con 0x7ff33c102a30 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338010690 con 0x7ff33c102a30 2024-04-25T13:02:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.007+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a0740 con 0x7ff33c102a30 2024-04-25T13:02:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.008+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c0fdf30 con 0x7ff33c102a30 2024-04-25T13:02:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.009+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff3380046c0 con 0x7ff33c102a30 2024-04-25T13:02:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.010+0000 7ff3337fe640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff3280a3bb0 0x7ff3280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.010+0000 7ff341acb640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff3280a3bb0 0x7ff3280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.011+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7ff3380d0760 con 0x7ff33c102a30 2024-04-25T13:02:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.011+0000 7ff341acb640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff3280a3bb0 0x7ff3280a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009230 tx=0x7ff32c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.015+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff338053ff0 con 0x7ff33c102a30 2024-04-25T13:02:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.202+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff33c02d5c0 con 0x7ff33c102a30 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/881900434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3796509073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:43.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:42 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/881900434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3796509073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:42 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/881900434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3796509073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:43.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:43.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:42 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.889+0000 7ff3337fe640 1 -- 172.21.15.38:0/2513467201 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7ff33809d3d0 con 0x7ff33c102a30 2024-04-25T13:02:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T13:02:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff3280a3bb0 msgr2=0x7ff3280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff3280a3bb0 0x7ff3280a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009230 tx=0x7ff32c031400 comp rx=0 tx=0).stop 2024-04-25T13:02:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 msgr2=0x7ff33c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 0x7ff33c1983b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff33800e9f0 tx=0x7ff33800edf0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 shutdown_connections 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff3280a3bb0 0x7ff3280a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff33c107c90 0x7ff33c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff33c105360 0x7ff33c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 --2- 172.21.15.38:0/2513467201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff33c102a30 0x7ff33c1983b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 >> 172.21.15.38:0/2513467201 conn(0x7ff33c0fc440 msgr2=0x7ff33c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 shutdown_connections 2024-04-25T13:02:43.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:43.892+0000 7ff344530640 1 -- 172.21.15.38:0/2513467201 wait complete. 2024-04-25T13:02:43.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:02:44.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- 172.21.15.38:0/2919550191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb924107d00 msgr2=0x7fb92410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:44.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- 172.21.15.38:0/2919550191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb924107d00 0x7fb92410a0f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009ca0 tx=0x7fb91c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- 172.21.15.38:0/2919550191 shutdown_connections 2024-04-25T13:02:44.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- 172.21.15.38:0/2919550191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb924107d00 0x7fb92410a0f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- 172.21.15.38:0/2919550191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9241053d0 0x7fb9241077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- 172.21.15.38:0/2919550191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb924102aa0 0x7fb924104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- 172.21.15.38:0/2919550191 >> 172.21.15.38:0/2919550191 conn(0x7fb9240fc410 msgr2=0x7fb9240fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:44.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- 172.21.15.38:0/2919550191 shutdown_connections 2024-04-25T13:02:44.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- 172.21.15.38:0/2919550191 wait complete. 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 Processor -- start 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- start start 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb924102aa0 0x7fb9241982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 0x7fb924198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb924107d00 0x7fb92419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410dc20 con 0x7fb9241053d0 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410daa0 con 0x7fb924102aa0 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.022+0000 7fb92a02a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410dda0 con 0x7fb924107d00 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 0x7fb924198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 0x7fb924198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51236/0 (socket says 172.21.15.38:51236) 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 -- 172.21.15.38:0/1900096509 learned_addr learned my addr 172.21.15.38:0/1900096509 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:44.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb9237fe640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb924102aa0 0x7fb9241982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb923fff640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb924107d00 0x7fb92419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 -- 172.21.15.38:0/1900096509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb924107d00 msgr2=0x7fb92419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb924107d00 0x7fb92419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 -- 172.21.15.38:0/1900096509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb924102aa0 msgr2=0x7fb9241982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb924102aa0 0x7fb9241982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 -- 172.21.15.38:0/1900096509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419ff10 con 0x7fb9241053d0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb91bfff640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 0x7fb924198830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb90c005fd0 tx=0x7fb90c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c017070 con 0x7fb9241053d0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb90c009c30 con 0x7fb9241053d0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241a0200 con 0x7fb9241053d0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c003de0 con 0x7fb9241053d0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.023+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a0750 con 0x7fb9241053d0 2024-04-25T13:02:44.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.024+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9240fdf00 con 0x7fb9241053d0 2024-04-25T13:02:44.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.025+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb90c003f80 con 0x7fb9241053d0 2024-04-25T13:02:44.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.026+0000 7fb9217fa640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb9100a3bb0 0x7fb9100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:44.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.026+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fb90c0d05c0 con 0x7fb9241053d0 2024-04-25T13:02:44.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.026+0000 7fb9237fe640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb9100a3bb0 0x7fb9100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:44.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.026+0000 7fb9237fe640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb9100a3bb0 0x7fb9100a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb914009bd0 tx=0x7fb914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:44.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.030+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb90c020030 con 0x7fb9241053d0 2024-04-25T13:02:44.052 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:43 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3796509073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:02:44.052 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:43 smithi104 ceph-mon[32726]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:02:44.052 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:43 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:44.052 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:43 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2513467201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:02:44.067 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:43 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3796509073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:02:44.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:43 smithi178 ceph-mon[33233]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:02:44.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:43 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:44.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:43 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2513467201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:02:44.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:43 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3796509073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:02:44.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:43 smithi038 ceph-mon[29400]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:02:44.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:43 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:02:44.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:43 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2513467201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:02:44.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.260+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb924062bc0 con 0x7fb9241053d0 2024-04-25T13:02:44.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.892+0000 7fb9217fa640 1 -- 172.21.15.38:0/1900096509 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb90c009de0 con 0x7fb9241053d0 2024-04-25T13:02:44.890 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb9100a3bb0 msgr2=0x7fb9100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb9100a3bb0 0x7fb9100a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb914009bd0 tx=0x7fb914005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 msgr2=0x7fb924198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 0x7fb924198830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb90c005fd0 tx=0x7fb90c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 shutdown_connections 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb9100a3bb0 0x7fb9100a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.894+0000 7fb92a02a640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb924107d00 0x7fb92419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.895+0000 7fb92a02a640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9241053d0 0x7fb924198830 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.895+0000 7fb92a02a640 1 --2- 172.21.15.38:0/1900096509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb924102aa0 0x7fb9241982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:44.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.895+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 >> 172.21.15.38:0/1900096509 conn(0x7fb9240fc410 msgr2=0x7fb9241034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:44.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.895+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 shutdown_connections 2024-04-25T13:02:44.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:44.895+0000 7fb92a02a640 1 -- 172.21.15.38:0/1900096509 wait complete. 2024-04-25T13:02:44.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3134665075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 msgr2=0x7f1ba01077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3134665075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba01077f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009900 tx=0x7f1b9c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3134665075 shutdown_connections 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3134665075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0107d30 0x7f1ba010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3134665075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba01077f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3134665075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ba0102ad0 0x7f1ba0104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3134665075 >> 172.21.15.38:0/3134665075 conn(0x7f1ba00fc440 msgr2=0x7f1ba00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3134665075 shutdown_connections 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.011+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3134665075 wait complete. 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 Processor -- start 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 -- start start 2024-04-25T13:02:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0102ad0 0x7f1ba0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ba0107d30 0x7f1ba019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010dc30 con 0x7f1ba0105400 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010dab0 con 0x7f1ba0102ad0 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba84cb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010ddb0 con 0x7f1ba0107d30 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba5a66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba5a66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51264/0 (socket says 172.21.15.38:51264) 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba5a66640 1 -- 172.21.15.38:0/3482914664 learned_addr learned my addr 172.21.15.38:0/3482914664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba6267640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0102ad0 0x7f1ba0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba6a68640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ba0107d30 0x7f1ba019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba6267640 1 -- 172.21.15.38:0/3482914664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ba0107d30 msgr2=0x7f1ba019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.012+0000 7f1ba6267640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ba0107d30 0x7f1ba019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba6267640 1 -- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 msgr2=0x7f1ba0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba6267640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba6267640 1 -- 172.21.15.38:0/3482914664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba019ff40 con 0x7f1ba0102ad0 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba5a66640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba6267640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0102ad0 0x7f1ba0198320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c970 tx=0x7f1b9000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90007bb0 con 0x7f1ba0102ad0 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba01a0230 con 0x7f1ba0102ad0 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1b90007d50 con 0x7f1ba0102ad0 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000f450 con 0x7f1ba0102ad0 2024-04-25T13:02:45.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.013+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba01a0730 con 0x7f1ba0102ad0 2024-04-25T13:02:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.014+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba00fdf30 con 0x7f1ba0102ad0 2024-04-25T13:02:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.015+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b90004f10 con 0x7f1ba0102ad0 2024-04-25T13:02:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.016+0000 7f1b977fe640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1b8c0a3bb0 0x7f1b8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.016+0000 7f1ba5a66640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1b8c0a3bb0 0x7f1b8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.016+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1b900cffb0 con 0x7f1ba0102ad0 2024-04-25T13:02:45.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.016+0000 7f1ba5a66640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1b8c0a3bb0 0x7f1b8c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0199830 tx=0x7f1b9c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:45.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.020+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b90014030 con 0x7f1ba0102ad0 2024-04-25T13:02:45.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.208+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ba0062b50 con 0x7f1ba0102ad0 2024-04-25T13:02:45.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.209+0000 7f1b977fe640 1 -- 172.21.15.38:0/3482914664 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22185 (secure 0 0 0) 0x7f1b900816c0 con 0x7f1ba0102ad0 2024-04-25T13:02:45.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:45.208 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":85,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:02:44.261380+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.211+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1b8c0a3bb0 msgr2=0x7f1b8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.211+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1b8c0a3bb0 0x7f1b8c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0199830 tx=0x7f1b9c031510 comp rx=0 tx=0).stop 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.211+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0102ad0 msgr2=0x7f1ba0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.211+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0102ad0 0x7f1ba0198320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c970 tx=0x7f1b9000ce30 comp rx=0 tx=0).stop 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 shutdown_connections 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1b8c0a3bb0 0x7f1b8c0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ba0107d30 0x7f1ba019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba0105400 0x7f1ba0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 --2- 172.21.15.38:0/3482914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ba0102ad0 0x7f1ba0198320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 >> 172.21.15.38:0/3482914664 conn(0x7f1ba00fc440 msgr2=0x7f1ba01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:45.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 shutdown_connections 2024-04-25T13:02:45.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.212+0000 7f1ba84cb640 1 -- 172.21.15.38:0/3482914664 wait complete. 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2513467201' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1900096509' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:02:45.220 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:45.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:44 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2513467201' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1900096509' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:45.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:45.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:44 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:02:45.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2513467201' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1900096509' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:02:45.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:02:45.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:44 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.326+0000 7f040d635640 1 -- 172.21.15.38:0/248775848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 msgr2=0x7f04081024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.326+0000 7f040d635640 1 --2- 172.21.15.38:0/248775848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f04081024c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009900 tx=0x7f03fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- 172.21.15.38:0/248775848 shutdown_connections 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 --2- 172.21.15.38:0/248775848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408102a00 0x7f0408109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 --2- 172.21.15.38:0/248775848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f04081024c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 --2- 172.21.15.38:0/248775848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0408100e60 0x7f0408101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- 172.21.15.38:0/248775848 >> 172.21.15.38:0/248775848 conn(0x7f04080fc440 msgr2=0x7f04080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- 172.21.15.38:0/248775848 shutdown_connections 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- 172.21.15.38:0/248775848 wait complete. 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 Processor -- start 2024-04-25T13:02:45.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- start start 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408100e60 0x7f0408198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f0408198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0408102a00 0x7f040819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f040810cb50 con 0x7f0408102040 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f040810c9d0 con 0x7f0408100e60 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.327+0000 7f040d635640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f040810ccd0 con 0x7f0408102a00 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f0408198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f0408198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51286/0 (socket says 172.21.15.38:51286) 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 -- 172.21.15.38:0/1750389020 learned_addr learned my addr 172.21.15.38:0/1750389020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f0406ffd640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408100e60 0x7f0408198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04077fe640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0408102a00 0x7f040819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 -- 172.21.15.38:0/1750389020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0408102a00 msgr2=0x7f040819f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0408102a00 0x7f040819f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 -- 172.21.15.38:0/1750389020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408100e60 msgr2=0x7f0408198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408100e60 0x7f0408198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 -- 172.21.15.38:0/1750389020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040819ff60 con 0x7f0408102040 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f0406ffd640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408100e60 0x7f0408198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f04067fc640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f0408198880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005940 tx=0x7f03fc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc03f030 con 0x7f0408102040 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03fc003cd0 con 0x7f0408102040 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04081a01f0 con 0x7f0408102040 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc035070 con 0x7f0408102040 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.328+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04081a06e0 con 0x7f0408102040 2024-04-25T13:02:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.329+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04081a0940 con 0x7f0408102040 2024-04-25T13:02:45.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.330+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f03fc042030 con 0x7f0408102040 2024-04-25T13:02:45.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.331+0000 7f03effff640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f03f00a3bb0 0x7f03f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.331+0000 7f0406ffd640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f03f00a3bb0 0x7f03f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.331+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f03fc0f0970 con 0x7f0408102040 2024-04-25T13:02:45.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.331+0000 7f0406ffd640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f03f00a3bb0 0x7f03f00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03f80058c0 tx=0x7f03f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:45.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.336+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03fc0bd4a0 con 0x7f0408102040 2024-04-25T13:02:45.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.526+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04081a1040 con 0x7f0408102040 2024-04-25T13:02:45.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.527+0000 7f03effff640 1 -- 172.21.15.38:0/1750389020 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22185 (secure 0 0 0) 0x7f03fc0bcda0 con 0x7f0408102040 2024-04-25T13:02:45.526 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:45.526 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":85,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:02:44.261380+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f03f00a3bb0 msgr2=0x7f03f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f03f00a3bb0 0x7f03f00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03f80058c0 tx=0x7f03f8005850 comp rx=0 tx=0).stop 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 msgr2=0x7f0408198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f0408198880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005940 tx=0x7f03fc0043b0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 shutdown_connections 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f03f00a3bb0 0x7f03f00a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0408102a00 0x7f040819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0408102040 0x7f0408198880 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 --2- 172.21.15.38:0/1750389020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0408100e60 0x7f0408198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.529+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 >> 172.21.15.38:0/1750389020 conn(0x7f04080fc440 msgr2=0x7f040806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.530+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 shutdown_connections 2024-04-25T13:02:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.530+0000 7f040d635640 1 -- 172.21.15.38:0/1750389020 wait complete. 2024-04-25T13:02:45.539 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:02:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 -- 172.21.15.38:0/3517240681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78100e60 msgr2=0x7f0d78101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 --2- 172.21.15.38:0/3517240681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78100e60 0x7f0d78101280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009900 tx=0x7f0d6802a410 comp rx=0 tx=0).stop 2024-04-25T13:02:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 -- 172.21.15.38:0/3517240681 shutdown_connections 2024-04-25T13:02:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 --2- 172.21.15.38:0/3517240681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d78102a00 0x7f0d78109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 --2- 172.21.15.38:0/3517240681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102040 0x7f0d781024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 --2- 172.21.15.38:0/3517240681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78100e60 0x7f0d78101280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 -- 172.21.15.38:0/3517240681 >> 172.21.15.38:0/3517240681 conn(0x7f0d780fc440 msgr2=0x7f0d780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.645+0000 7f0d7d594640 1 -- 172.21.15.38:0/3517240681 shutdown_connections 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 -- 172.21.15.38:0/3517240681 wait complete. 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 Processor -- start 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 -- start start 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d78100e60 0x7f0d7806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 0x7f0d7819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102a00 0x7f0d7819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810cb80 con 0x7f0d78102040 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810ca00 con 0x7f0d78102a00 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d7d594640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810cd00 con 0x7f0d78100e60 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d767fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 0x7f0d7819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.646+0000 7f0d767fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 0x7f0d7819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51298/0 (socket says 172.21.15.38:51298) 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 -- 172.21.15.38:0/1059294510 learned_addr learned my addr 172.21.15.38:0/1059294510 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d777fe640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102a00 0x7f0d7819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d76ffd640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d78100e60 0x7f0d7806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 -- 172.21.15.38:0/1059294510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d78100e60 msgr2=0x7f0d7806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d78100e60 0x7f0d7806b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 -- 172.21.15.38:0/1059294510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102a00 msgr2=0x7f0d7819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102a00 0x7f0d7819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 -- 172.21.15.38:0/1059294510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7819ff50 con 0x7f0d78102040 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d777fe640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102a00 0x7f0d7819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d767fc640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 0x7f0d7819d020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000d950 tx=0x7f0d6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60017070 con 0x7f0d78102040 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d60009c30 con 0x7f0d78102040 2024-04-25T13:02:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781a01e0 con 0x7f0d78102040 2024-04-25T13:02:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000f6b0 con 0x7f0d78102040 2024-04-25T13:02:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.647+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781a0730 con 0x7f0d78102040 2024-04-25T13:02:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.648+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d78108280 con 0x7f0d78102040 2024-04-25T13:02:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.649+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d600109c0 con 0x7f0d78102040 2024-04-25T13:02:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.650+0000 7f0d6ffff640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d640a3bb0 0x7f0d640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.650+0000 7f0d76ffd640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d640a3bb0 0x7f0d640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.650+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0d600d0030 con 0x7f0d78102040 2024-04-25T13:02:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.650+0000 7f0d76ffd640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d640a3bb0 0x7f0d640a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006010 tx=0x7f0d68031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.654+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d60009de0 con 0x7f0d78102040 2024-04-25T13:02:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:45.883+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0d78062b50 con 0x7f0d78102040 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1900096509' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3482914664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1750389020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] up:active 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:02:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:45 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1059294510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1900096509' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3482914664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1750389020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] up:active 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:02:46.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1059294510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:02:46.323 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: 2024-04-25T13:02:45.865+0000 7f0e80a22700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:02:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1900096509' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:02:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:02:46.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3482914664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1750389020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] up:active 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:02:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:45 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1059294510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:02:46.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.732+0000 7f0d6ffff640 1 -- 172.21.15.38:0/1059294510 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f0d6009cac0 con 0x7f0d78102040 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d640a3bb0 msgr2=0x7f0d640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d640a3bb0 0x7f0d640a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006010 tx=0x7f0d68031510 comp rx=0 tx=0).stop 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 msgr2=0x7f0d7819d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 0x7f0d7819d020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000d950 tx=0x7f0d6000de10 comp rx=0 tx=0).stop 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 shutdown_connections 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d640a3bb0 0x7f0d640a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d78102a00 0x7f0d7819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d78102040 0x7f0d7819d020 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 --2- 172.21.15.38:0/1059294510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d78100e60 0x7f0d7806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 >> 172.21.15.38:0/1059294510 conn(0x7f0d780fc440 msgr2=0x7f0d78105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 shutdown_connections 2024-04-25T13:02:46.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.735+0000 7f0d7d594640 1 -- 172.21.15.38:0/1059294510 wait complete. 2024-04-25T13:02:46.746 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T13:02:46.746 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T13:02:46.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 -- 172.21.15.38:0/841877581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80105360 msgr2=0x7ffa80107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 --2- 172.21.15.38:0/841877581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80105360 0x7ffa80107750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009900 tx=0x7ffa7c02a410 comp rx=0 tx=0).stop 2024-04-25T13:02:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 -- 172.21.15.38:0/841877581 shutdown_connections 2024-04-25T13:02:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 --2- 172.21.15.38:0/841877581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80107c90 0x7ffa8010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 --2- 172.21.15.38:0/841877581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80105360 0x7ffa80107750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 --2- 172.21.15.38:0/841877581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa80102a30 0x7ffa80104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 -- 172.21.15.38:0/841877581 >> 172.21.15.38:0/841877581 conn(0x7ffa800fc440 msgr2=0x7ffa800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.860+0000 7ffa87b19640 1 -- 172.21.15.38:0/841877581 shutdown_connections 2024-04-25T13:02:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 -- 172.21.15.38:0/841877581 wait complete. 2024-04-25T13:02:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 Processor -- start 2024-04-25T13:02:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 -- start start 2024-04-25T13:02:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa80102a30 0x7ffa801982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80105360 0x7ffa80198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 0x7ffa8019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8010dc10 con 0x7ffa80107c90 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8010da90 con 0x7ffa80102a30 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa87b19640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8010dd90 con 0x7ffa80105360 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa860b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 0x7ffa8019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa860b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 0x7ffa8019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51324/0 (socket says 172.21.15.38:51324) 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.861+0000 7ffa860b6640 1 -- 172.21.15.38:0/2735259787 learned_addr learned my addr 172.21.15.38:0/2735259787 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa850b4640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80105360 0x7ffa80198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa858b5640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa80102a30 0x7ffa801982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa860b6640 1 -- 172.21.15.38:0/2735259787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80105360 msgr2=0x7ffa80198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa860b6640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80105360 0x7ffa80198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa860b6640 1 -- 172.21.15.38:0/2735259787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa80102a30 msgr2=0x7ffa801982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa860b6640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa80102a30 0x7ffa801982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa860b6640 1 -- 172.21.15.38:0/2735259787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8019fee0 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa850b4640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80105360 0x7ffa80198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa860b6640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 0x7ffa8019f870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800cce0 tx=0x7ffa7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800f450 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ffa7800fac0 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa801a01d0 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78018be0 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.862+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801a06f0 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.863+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa800fdf30 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.864+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa7800fc70 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.865+0000 7ffa76ffd640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffa6c0a3c30 0x7ffa6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.865+0000 7ffa858b5640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffa6c0a3c30 0x7ffa6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.865+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffa780d09b0 con 0x7ffa80107c90 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.865+0000 7ffa858b5640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffa6c0a3c30 0x7ffa6c0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005fd0 tx=0x7ffa70005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:46.869+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa78014030 con 0x7ffa80107c90 2024-04-25T13:02:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:46 smithi038 ceph-mon[29400]: pgmap v290: 97 pgs: 12 creating+peering, 17 active+clean, 68 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-25T13:02:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:46 smithi038 ceph-mon[29400]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:02:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:46 smithi038 ceph-mon[29400]: Cluster is now healthy 2024-04-25T13:02:47.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:46 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1059294510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:02:47.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:46 smithi038 ceph-mon[29400]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:47.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.097+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ffa80062b50 con 0x7ffa80107c90 2024-04-25T13:02:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:46 smithi178 ceph-mon[33233]: pgmap v290: 97 pgs: 12 creating+peering, 17 active+clean, 68 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-25T13:02:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:46 smithi178 ceph-mon[33233]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:02:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:46 smithi178 ceph-mon[33233]: Cluster is now healthy 2024-04-25T13:02:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:46 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1059294510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:02:47.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:46 smithi178 ceph-mon[33233]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:47.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:46 smithi104 ceph-mon[32726]: pgmap v290: 97 pgs: 12 creating+peering, 17 active+clean, 68 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-25T13:02:47.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:46 smithi104 ceph-mon[32726]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:02:47.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:46 smithi104 ceph-mon[32726]: Cluster is now healthy 2024-04-25T13:02:47.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:46 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1059294510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:02:47.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:46 smithi104 ceph-mon[32726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:47.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.917+0000 7ffa76ffd640 1 -- 172.21.15.38:0/2735259787 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7ffa7809d4e0 con 0x7ffa80107c90 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffa6c0a3c30 msgr2=0x7ffa6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffa6c0a3c30 0x7ffa6c0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005fd0 tx=0x7ffa70005850 comp rx=0 tx=0).stop 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 msgr2=0x7ffa8019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 0x7ffa8019f870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800cce0 tx=0x7ffa7800a620 comp rx=0 tx=0).stop 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 shutdown_connections 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffa6c0a3c30 0x7ffa6c0a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:47.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa80107c90 0x7ffa8019f870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa80105360 0x7ffa80198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 --2- 172.21.15.38:0/2735259787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa80102a30 0x7ffa801982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 >> 172.21.15.38:0/2735259787 conn(0x7ffa800fc440 msgr2=0x7ffa80103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 shutdown_connections 2024-04-25T13:02:47.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:47.921+0000 7ffa87b19640 1 -- 172.21.15.38:0/2735259787 wait complete. 2024-04-25T13:02:47.935 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:02:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 -- 172.21.15.38:0/993517126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80107c90 msgr2=0x7eff8010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 --2- 172.21.15.38:0/993517126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80107c90 0x7eff8010a0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7eff78009ca0 tx=0x7eff7802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 -- 172.21.15.38:0/993517126 shutdown_connections 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 --2- 172.21.15.38:0/993517126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80107c90 0x7eff8010a0e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 --2- 172.21.15.38:0/993517126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80105360 0x7eff80107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 --2- 172.21.15.38:0/993517126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff80102a30 0x7eff80104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 -- 172.21.15.38:0/993517126 >> 172.21.15.38:0/993517126 conn(0x7eff800fc440 msgr2=0x7eff800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 -- 172.21.15.38:0/993517126 shutdown_connections 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 -- 172.21.15.38:0/993517126 wait complete. 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.047+0000 7eff85279640 1 Processor -- start 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 -- start start 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80102a30 0x7eff80198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 0x7eff80198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff80107c90 0x7eff8019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010dd10 con 0x7eff80105360 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010db90 con 0x7eff80102a30 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff85279640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010de90 con 0x7eff80107c90 2024-04-25T13:02:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 0x7eff80198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 0x7eff80198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51336/0 (socket says 172.21.15.38:51336) 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 -- 172.21.15.38:0/1210926294 learned_addr learned my addr 172.21.15.38:0/1210926294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7f7fe640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff80107c90 0x7eff8019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7effd640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80102a30 0x7eff80198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 -- 172.21.15.38:0/1210926294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff80107c90 msgr2=0x7eff8019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff80107c90 0x7eff8019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 -- 172.21.15.38:0/1210926294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80102a30 msgr2=0x7eff80198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80102a30 0x7eff80198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 -- 172.21.15.38:0/1210926294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff801a0030 con 0x7eff80105360 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7effd640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80102a30 0x7eff80198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.048+0000 7eff7e7fc640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 0x7eff80198950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b450 tx=0x7eff7400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.049+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74015020 con 0x7eff80105360 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.049+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff801a0320 con 0x7eff80105360 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.049+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff74011d60 con 0x7eff80105360 2024-04-25T13:02:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.049+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7400f340 con 0x7eff80105360 2024-04-25T13:02:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.049+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff801a0940 con 0x7eff80105360 2024-04-25T13:02:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.050+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff800fdf30 con 0x7eff80105360 2024-04-25T13:02:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.051+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff74018020 con 0x7eff80105360 2024-04-25T13:02:48.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.052+0000 7eff67fff640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7eff680a3bb0 0x7eff680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:48.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.052+0000 7eff7effd640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7eff680a3bb0 0x7eff680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:48.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.052+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7eff740d03f0 con 0x7eff80105360 2024-04-25T13:02:48.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.052+0000 7eff7effd640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7eff680a3bb0 0x7eff680a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.055+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff7401f020 con 0x7eff80105360 2024-04-25T13:02:48.225 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:47 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2735259787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T13:02:48.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:47 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2735259787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T13:02:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.281+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7eff8002d5c0 con 0x7eff80105360 2024-04-25T13:02:48.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:47 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2735259787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T13:02:48.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.929+0000 7eff67fff640 1 -- 172.21.15.38:0/1210926294 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7eff7400d040 con 0x7eff80105360 2024-04-25T13:02:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7eff680a3bb0 msgr2=0x7eff680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7eff680a3bb0 0x7eff680a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70008040 comp rx=0 tx=0).stop 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 msgr2=0x7eff80198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 0x7eff80198950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b450 tx=0x7eff7400b910 comp rx=0 tx=0).stop 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 shutdown_connections 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7eff680a3bb0 0x7eff680a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff80107c90 0x7eff8019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff80105360 0x7eff80198950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 --2- 172.21.15.38:0/1210926294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff80102a30 0x7eff80198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 >> 172.21.15.38:0/1210926294 conn(0x7eff800fc440 msgr2=0x7eff80103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 shutdown_connections 2024-04-25T13:02:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:48.932+0000 7eff85279640 1 -- 172.21.15.38:0/1210926294 wait complete. 2024-04-25T13:02:48.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:02:49.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/2571379196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4107d30 msgr2=0x7f7ea410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/2571379196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4107d30 0x7f7ea410a120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009ca0 tx=0x7f7e9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/2571379196 shutdown_connections 2024-04-25T13:02:49.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/2571379196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4107d30 0x7f7ea410a120 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/2571379196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 0x7f7ea41077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/2571379196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/2571379196 >> 172.21.15.38:0/2571379196 conn(0x7f7ea40fc440 msgr2=0x7f7ea40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/2571379196 shutdown_connections 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.060+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/2571379196 wait complete. 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 Processor -- start 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 -- start start 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 0x7f7ea4100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea4107d30 0x7f7ea4100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea410dc30 con 0x7f7ea4102ad0 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea410dab0 con 0x7f7ea4107d30 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7eaa0ef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea410ddb0 con 0x7f7ea4105400 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.061+0000 7f7ea37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4102440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51358/0 (socket says 172.21.15.38:51358) 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 -- 172.21.15.38:0/3318089149 learned_addr learned my addr 172.21.15.38:0/3318089149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea3fff640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea4107d30 0x7f7ea4100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea2ffd640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 0x7f7ea4100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 -- 172.21.15.38:0/3318089149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 msgr2=0x7f7ea4100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 0x7f7ea4100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 -- 172.21.15.38:0/3318089149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea4107d30 msgr2=0x7f7ea4100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea4107d30 0x7f7ea4100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 -- 172.21.15.38:0/3318089149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea4101850 con 0x7f7ea4102ad0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea2ffd640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 0x7f7ea4100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea37fe640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4102440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9f0 tx=0x7f7e9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cd80 con 0x7f7ea4102ad0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7e94010900 con 0x7f7ea4102ad0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41a23c0 con 0x7f7ea4102ad0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940189c0 con 0x7f7ea4102ad0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.062+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41a2910 con 0x7f7ea4102ad0 2024-04-25T13:02:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.063+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea40fdf30 con 0x7f7ea4102ad0 2024-04-25T13:02:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.064+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e94010ab0 con 0x7f7ea4102ad0 2024-04-25T13:02:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.065+0000 7f7ea0ff9640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7e8c0a3bb0 0x7f7e8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.065+0000 7f7ea2ffd640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7e8c0a3bb0 0x7f7e8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.065+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7e940d0610 con 0x7f7ea4102ad0 2024-04-25T13:02:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.065+0000 7f7ea2ffd640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7e8c0a3bb0 0x7f7e8c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005fd0 tx=0x7f7e98005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:49.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.069+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e9409d140 con 0x7f7ea4102ad0 2024-04-25T13:02:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: pgmap v292: 97 pgs: 12 creating+peering, 77 active+clean, 8 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-25T13:02:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:02:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2735259787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T13:02:49.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:02:49.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:49.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:02:49.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:02:49.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:48 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1210926294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:02:49.240 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: pgmap v292: 97 pgs: 12 creating+peering, 77 active+clean, 8 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-25T13:02:49.240 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:02:49.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2735259787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T13:02:49.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:02:49.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:49.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:02:49.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:02:49.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:48 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1210926294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:02:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.297+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7ea402d5c0 con 0x7f7ea4102ad0 2024-04-25T13:02:49.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: pgmap v292: 97 pgs: 12 creating+peering, 77 active+clean, 8 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2735259787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:02:49.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1210926294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:02:49.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.952+0000 7f7ea0ff9640 1 -- 172.21.15.38:0/3318089149 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f7e9409ca40 con 0x7f7ea4102ad0 2024-04-25T13:02:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7e8c0a3bb0 msgr2=0x7f7e8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7e8c0a3bb0 0x7f7e8c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005fd0 tx=0x7f7e98005ea0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 msgr2=0x7f7ea4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4102440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9f0 tx=0x7f7e9400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 shutdown_connections 2024-04-25T13:02:49.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7e8c0a3bb0 0x7f7e8c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea4107d30 0x7f7ea4100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea4105400 0x7f7ea4100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 --2- 172.21.15.38:0/3318089149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea4102ad0 0x7f7ea4102440 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:49.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.955+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 >> 172.21.15.38:0/3318089149 conn(0x7f7ea40fc440 msgr2=0x7f7ea4103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:49.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.956+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 shutdown_connections 2024-04-25T13:02:49.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:49.956+0000 7f7eaa0ef640 1 -- 172.21.15.38:0/3318089149 wait complete. 2024-04-25T13:02:49.971 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T13:02:49.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:02:50.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 -- 172.21.15.38:0/3865678179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c102040 msgr2=0x7f972c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 --2- 172.21.15.38:0/3865678179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c102040 0x7f972c1024c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9720009900 tx=0x7f972002a410 comp rx=0 tx=0).stop 2024-04-25T13:02:50.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 -- 172.21.15.38:0/3865678179 shutdown_connections 2024-04-25T13:02:50.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 --2- 172.21.15.38:0/3865678179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f972c102a00 0x7f972c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 --2- 172.21.15.38:0/3865678179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c102040 0x7f972c1024c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 --2- 172.21.15.38:0/3865678179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f972c100e60 0x7f972c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 -- 172.21.15.38:0/3865678179 >> 172.21.15.38:0/3865678179 conn(0x7f972c0fc440 msgr2=0x7f972c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 -- 172.21.15.38:0/3865678179 shutdown_connections 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 -- 172.21.15.38:0/3865678179 wait complete. 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 Processor -- start 2024-04-25T13:02:50.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.085+0000 7f9732368640 1 -- start start 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 0x7f972c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f972c102040 0x7f972c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f972c102a00 0x7f972c19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10dc40 con 0x7f972c100e60 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10dac0 con 0x7f972c102a00 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10ddc0 con 0x7f972c102040 2024-04-25T13:02:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 0x7f972c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 0x7f972c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37480/0 (socket says 172.21.15.38:37480) 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 -- 172.21.15.38:0/1409537571 learned_addr learned my addr 172.21.15.38:0/1409537571 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9730905640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f972c102a00 0x7f972c19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972b7fe640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f972c102040 0x7f972c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 -- 172.21.15.38:0/1409537571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f972c102040 msgr2=0x7f972c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f972c102040 0x7f972c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 -- 172.21.15.38:0/1409537571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f972c102a00 msgr2=0x7f972c19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f972c102a00 0x7f972c19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 -- 172.21.15.38:0/1409537571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c19ff00 con 0x7f972c100e60 2024-04-25T13:02:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f972bfff640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 0x7f972c1982e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f971c00e9c0 tx=0x7f971c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c00cd80 con 0x7f972c100e60 2024-04-25T13:02:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c1a01f0 con 0x7f972c100e60 2024-04-25T13:02:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f971c0035f0 con 0x7f972c100e60 2024-04-25T13:02:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c010690 con 0x7f972c100e60 2024-04-25T13:02:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.086+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c1a0760 con 0x7f972c100e60 2024-04-25T13:02:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.088+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972c108200 con 0x7f972c100e60 2024-04-25T13:02:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.089+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f971c0037a0 con 0x7f972c100e60 2024-04-25T13:02:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.090+0000 7f97297fa640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f97140a3bb0 0x7f97140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.090+0000 7f972b7fe640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f97140a3bb0 0x7f97140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.090+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f971c0d0710 con 0x7f972c100e60 2024-04-25T13:02:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.090+0000 7f972b7fe640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f97140a3bb0 0x7f97140a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9720000c00 tx=0x7f9720031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.094+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f971c09d1a0 con 0x7f972c100e60 2024-04-25T13:02:50.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1210926294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:02:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] up:active 2024-04-25T13:02:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:02:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:02:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T13:02:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T13:02:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:49 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3318089149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:02:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.309+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f972c062390 con 0x7f97140a3bb0 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1210926294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] up:active 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T13:02:50.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:49 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3318089149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:02:50.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1210926294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:02:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] up:active 2024-04-25T13:02:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:02:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T13:02:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T13:02:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T13:02:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:49 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3318089149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:02:50.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.328+0000 7f97297fa640 1 -- 172.21.15.38:0/1409537571 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f972c062390 con 0x7f97140a3bb0 2024-04-25T13:02:50.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.330+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f97140a3bb0 msgr2=0x7f97140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f97140a3bb0 0x7f97140a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9720000c00 tx=0x7f9720031510 comp rx=0 tx=0).stop 2024-04-25T13:02:50.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 msgr2=0x7f972c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 0x7f972c1982e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f971c00e9c0 tx=0x7f971c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 shutdown_connections 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f97140a3bb0 0x7f97140a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f972c102a00 0x7f972c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f972c102040 0x7f972c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 --2- 172.21.15.38:0/1409537571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f972c100e60 0x7f972c1982e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 >> 172.21.15.38:0/1409537571 conn(0x7f972c0fc440 msgr2=0x7f972c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 shutdown_connections 2024-04-25T13:02:50.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.331+0000 7f9732368640 1 -- 172.21.15.38:0/1409537571 wait complete. 2024-04-25T13:02:50.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:02:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 -- 172.21.15.38:0/3366234521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103bf0 msgr2=0x7f646410a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 --2- 172.21.15.38:0/3366234521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103bf0 0x7f646410a480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f645c009cd0 tx=0x7f645c02a8e0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 -- 172.21.15.38:0/3366234521 shutdown_connections 2024-04-25T13:02:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 --2- 172.21.15.38:0/3366234521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103bf0 0x7f646410a480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 --2- 172.21.15.38:0/3366234521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6464103230 0x7f64641036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 --2- 172.21.15.38:0/3366234521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6464100260 0x7f6464100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 -- 172.21.15.38:0/3366234521 >> 172.21.15.38:0/3366234521 conn(0x7f64640fbb90 msgr2=0x7f64640fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:50.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 -- 172.21.15.38:0/3366234521 shutdown_connections 2024-04-25T13:02:50.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 -- 172.21.15.38:0/3366234521 wait complete. 2024-04-25T13:02:50.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.449+0000 7f646c37d640 1 Processor -- start 2024-04-25T13:02:50.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 -- start start 2024-04-25T13:02:50.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6464100260 0x7f6464102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 0x7f64641011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6464103bf0 0x7f6464101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f646410dc00 con 0x7f6464103230 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f646410da80 con 0x7f6464103bf0 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f646410dd80 con 0x7f6464100260 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 0x7f64641011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 0x7f64641011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37516/0 (socket says 172.21.15.38:37516) 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 -- 172.21.15.38:0/3248261862 learned_addr learned my addr 172.21.15.38:0/3248261862 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646a91a640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6464103bf0 0x7f6464101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646a119640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6464100260 0x7f6464102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 -- 172.21.15.38:0/3248261862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6464100260 msgr2=0x7f6464102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6464100260 0x7f6464102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 -- 172.21.15.38:0/3248261862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6464103bf0 msgr2=0x7f6464101720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6464103bf0 0x7f6464101720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 -- 172.21.15.38:0/3248261862 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6464101fb0 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f6469918640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 0x7f64641011e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f645400e9f0 tx=0x7f645400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645400cd80 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6454010900 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64641a2390 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.450+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540189c0 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.451+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a28e0 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.452+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6464109290 con 0x7f6464103230 2024-04-25T13:02:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.453+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6454010ab0 con 0x7f6464103230 2024-04-25T13:02:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.453+0000 7f645b7fe640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.453+0000 7f646a119640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.453+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f64540d05a0 con 0x7f6464103230 2024-04-25T13:02:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.454+0000 7f646a119640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64600058c0 tx=0x7f6460005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:50.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.458+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f645409d0d0 con 0x7f6464103230 2024-04-25T13:02:50.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:02:50.314+0000 7f1d807a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:02:50.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:02:50.314+0000 7f1d807a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:02:50.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:02:50.314+0000 7f1d807a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:02:50.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:02:50.314+0000 7f1d807a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:02:50.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:02:50.314+0000 7f1d807a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:02:50.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.672+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6464062420 con 0x7f64500a3bb0 2024-04-25T13:02:50.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.703+0000 7f645b7fe640 1 -- 172.21.15.38:0/3248261862 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6464062420 con 0x7f64500a3bb0 2024-04-25T13:02:50.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 msgr2=0x7f64500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64600058c0 tx=0x7f6460005850 comp rx=0 tx=0).stop 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 msgr2=0x7f64641011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 0x7f64641011e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f645400e9f0 tx=0x7f645400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 shutdown_connections 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64500a3bb0 0x7f64500a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6464103bf0 0x7f6464101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6464103230 0x7f64641011e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 --2- 172.21.15.38:0/3248261862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6464100260 0x7f6464102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 >> 172.21.15.38:0/3248261862 conn(0x7f64640fbb90 msgr2=0x7f6464106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 shutdown_connections 2024-04-25T13:02:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.705+0000 7f646c37d640 1 -- 172.21.15.38:0/3248261862 wait complete. 2024-04-25T13:02:50.714 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 -- 172.21.15.38:0/1715354850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102a00 msgr2=0x7f6778109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 --2- 172.21.15.38:0/1715354850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102a00 0x7f6778109320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6770009ca0 tx=0x7f677002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 -- 172.21.15.38:0/1715354850 shutdown_connections 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 --2- 172.21.15.38:0/1715354850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102a00 0x7f6778109320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 --2- 172.21.15.38:0/1715354850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778102040 0x7f67781024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 --2- 172.21.15.38:0/1715354850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778100e60 0x7f6778101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 -- 172.21.15.38:0/1715354850 >> 172.21.15.38:0/1715354850 conn(0x7f67780fc440 msgr2=0x7f67780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 -- 172.21.15.38:0/1715354850 shutdown_connections 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.823+0000 7f677d0ef640 1 -- 172.21.15.38:0/1715354850 wait complete. 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 Processor -- start 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 -- start start 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778100e60 0x7f6778104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 0x7f67781034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778102a00 0x7f67781039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f677810cb30 con 0x7f6778102040 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f677810c9b0 con 0x7f6778100e60 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677d0ef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f677810ccb0 con 0x7f6778102a00 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 0x7f67781034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f6776d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778100e60 0x7f6778104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 0x7f67781034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37534/0 (socket says 172.21.15.38:37534) 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677659c640 1 -- 172.21.15.38:0/3295503461 learned_addr learned my addr 172.21.15.38:0/3295503461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.824+0000 7f677759e640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778102a00 0x7f67781039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f6776d9d640 1 -- 172.21.15.38:0/3295503461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778102a00 msgr2=0x7f67781039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f6776d9d640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778102a00 0x7f67781039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f6776d9d640 1 -- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 msgr2=0x7f67781034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f6776d9d640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 0x7f67781034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f6776d9d640 1 -- 172.21.15.38:0/3295503461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778104280 con 0x7f6778100e60 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f677759e640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778102a00 0x7f67781039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f677659c640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 0x7f67781034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f6776d9d640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778100e60 0x7f6778104e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f676800c940 tx=0x7f676800ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768007bb0 con 0x7f6778100e60 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6768007d50 con 0x7f6778100e60 2024-04-25T13:02:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781a2330 con 0x7f6778100e60 2024-04-25T13:02:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768018780 con 0x7f6778100e60 2024-04-25T13:02:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.825+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781a28d0 con 0x7f6778100e60 2024-04-25T13:02:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.826+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67781a2bf0 con 0x7f6778100e60 2024-04-25T13:02:50.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.827+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f676800f450 con 0x7f6778100e60 2024-04-25T13:02:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.828+0000 7f675ffff640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f67600a3c30 0x7f67600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.828+0000 7f677659c640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f67600a3c30 0x7f67600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.828+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f67680d0620 con 0x7f6778100e60 2024-04-25T13:02:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.828+0000 7f677659c640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f67600a3c30 0x7f67600a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6778104be0 tx=0x7f676c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:50.832+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6768014030 con 0x7f6778100e60 2024-04-25T13:02:51.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.048+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f67780fe500 con 0x7f67600a3c30 2024-04-25T13:02:51.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.053+0000 7f675ffff640 1 -- 172.21.15.38:0/3295503461 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f67780fe500 con 0x7f67600a3c30 2024-04-25T13:02:51.051 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_0/79170f9e-e25f-4d9b-a71d-a3461c00a441 2024-04-25T13:02:51.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f67600a3c30 msgr2=0x7f67600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f67600a3c30 0x7f67600a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6778104be0 tx=0x7f676c005f90 comp rx=0 tx=0).stop 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778100e60 msgr2=0x7f6778104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778100e60 0x7f6778104e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f676800c940 tx=0x7f676800ce00 comp rx=0 tx=0).stop 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 shutdown_connections 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f67600a3c30 0x7f67600a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6778102a00 0x7f67781039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6778102040 0x7f67781034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 --2- 172.21.15.38:0/3295503461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6778100e60 0x7f6778104e70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 >> 172.21.15.38:0/3295503461 conn(0x7f67780fc440 msgr2=0x7f67780fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 shutdown_connections 2024-04-25T13:02:51.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.055+0000 7f677d0ef640 1 -- 172.21.15.38:0/3295503461 wait complete. 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: pgmap v293: 97 pgs: 6 creating+peering, 91 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3318089149' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] up:active 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] up:active 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: Cluster is now healthy 2024-04-25T13:02:51.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T13:02:51.065 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T13:02:51.065 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:50 smithi038 ceph-mon[29400]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T13:02:51.065 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 -- 172.21.15.38:0/3588300378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 msgr2=0x7f0d9c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 --2- 172.21.15.38:0/3588300378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c1012b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009900 tx=0x7f0d9402a410 comp rx=0 tx=0).stop 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 -- 172.21.15.38:0/3588300378 shutdown_connections 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 --2- 172.21.15.38:0/3588300378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c1029a0 0x7f0d9c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 --2- 172.21.15.38:0/3588300378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101fe0 0x7f0d9c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 --2- 172.21.15.38:0/3588300378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c1012b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 -- 172.21.15.38:0/3588300378 >> 172.21.15.38:0/3588300378 conn(0x7f0d9c0fc400 msgr2=0x7f0d9c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 -- 172.21.15.38:0/3588300378 shutdown_connections 2024-04-25T13:02:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.174+0000 7f0da400b640 1 -- 172.21.15.38:0/3588300378 wait complete. 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 Processor -- start 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 -- start start 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c101fe0 0x7f0d9c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c1029a0 0x7f0d9c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10cb50 con 0x7f0d9c100e90 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10c9d0 con 0x7f0d9c101fe0 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da400b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10ccd0 con 0x7f0d9c1029a0 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da1da7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da1da7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37560/0 (socket says 172.21.15.38:37560) 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da1da7640 1 -- 172.21.15.38:0/2638888544 learned_addr learned my addr 172.21.15.38:0/2638888544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da15a6640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c101fe0 0x7f0d9c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da1da7640 1 -- 172.21.15.38:0/2638888544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c1029a0 msgr2=0x7f0d9c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.175+0000 7f0da25a8640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c1029a0 0x7f0d9c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da1da7640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c1029a0 0x7f0d9c1a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da1da7640 1 -- 172.21.15.38:0/2638888544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c101fe0 msgr2=0x7f0d9c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da1da7640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c101fe0 0x7f0d9c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da1da7640 1 -- 172.21.15.38:0/2638888544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c1a2160 con 0x7f0d9c100e90 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da15a6640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c101fe0 0x7f0d9c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da1da7640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c069c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005ab0 tx=0x7f0d94032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9403f030 con 0x7f0d9c100e90 2024-04-25T13:02:51.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d94003ac0 con 0x7f0d9c100e90 2024-04-25T13:02:51.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1a2390 con 0x7f0d9c100e90 2024-04-25T13:02:51.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94035070 con 0x7f0d9c100e90 2024-04-25T13:02:51.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.176+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a28d0 con 0x7f0d9c100e90 2024-04-25T13:02:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.177+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c108270 con 0x7f0d9c100e90 2024-04-25T13:02:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.178+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d94042030 con 0x7f0d9c100e90 2024-04-25T13:02:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.179+0000 7f0d9affd640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d840a3c60 0x7f0d840a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.179+0000 7f0da15a6640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d840a3c60 0x7f0d840a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.179+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0d940f0ba0 con 0x7f0d9c100e90 2024-04-25T13:02:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.180+0000 7f0da15a6640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d840a3c60 0x7f0d840a6120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0699c0 tx=0x7f0d9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.184+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d940bd670 con 0x7f0d9c100e90 2024-04-25T13:02:51.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: pgmap v293: 97 pgs: 6 creating+peering, 91 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-04-25T13:02:51.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3318089149' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:02:51.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] up:active 2024-04-25T13:02:51.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T13:02:51.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T13:02:51.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-25T13:02:51.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-25T13:02:51.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] up:active 2024-04-25T13:02:51.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T13:02:51.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:02:51.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: Cluster is now healthy 2024-04-25T13:02:51.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T13:02:51.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T13:02:51.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:50 smithi178 ceph-mon[33233]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T13:02:51.323 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: 2024-04-25T13:02:50.865+0000 7f0e80a22700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T13:02:51.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: pgmap v293: 97 pgs: 6 creating+peering, 91 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3318089149' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] up:active 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] up:active 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: Cluster is now healthy 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T13:02:51.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T13:02:51.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:50 smithi104 ceph-mon[32726]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T13:02:51.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.403+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0d9c062420 con 0x7f0d840a3c60 2024-04-25T13:02:51.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.431+0000 7f0d9affd640 1 -- 172.21.15.38:0/2638888544 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d9c062420 con 0x7f0d840a3c60 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d840a3c60 msgr2=0x7f0d840a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d840a3c60 0x7f0d840a6120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0699c0 tx=0x7f0d9000b410 comp rx=0 tx=0).stop 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 msgr2=0x7f0d9c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c069c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005ab0 tx=0x7f0d94032bc0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 shutdown_connections 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0d840a3c60 0x7f0d840a6120 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c1029a0 0x7f0d9c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c101fe0 0x7f0d9c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 --2- 172.21.15.38:0/2638888544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d9c100e90 0x7f0d9c069c50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 >> 172.21.15.38:0/2638888544 conn(0x7f0d9c0fc400 msgr2=0x7f0d9c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 shutdown_connections 2024-04-25T13:02:51.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.433+0000 7f0da400b640 1 -- 172.21.15.38:0/2638888544 wait complete. 2024-04-25T13:02:51.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:02:51.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 -- 172.21.15.38:0/1345045443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba0101fe0 msgr2=0x7f7ba0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 --2- 172.21.15.38:0/1345045443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba0101fe0 0x7f7ba0102460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009900 tx=0x7f7b9002a3f0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 -- 172.21.15.38:0/1345045443 shutdown_connections 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 --2- 172.21.15.38:0/1345045443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba01029a0 0x7f7ba01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 --2- 172.21.15.38:0/1345045443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba0101fe0 0x7f7ba0102460 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 --2- 172.21.15.38:0/1345045443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 0x7f7ba01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 -- 172.21.15.38:0/1345045443 >> 172.21.15.38:0/1345045443 conn(0x7f7ba00fc400 msgr2=0x7f7ba00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 -- 172.21.15.38:0/1345045443 shutdown_connections 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.551+0000 7f7ba7431640 1 -- 172.21.15.38:0/1345045443 wait complete. 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 Processor -- start 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 -- start start 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 0x7f7ba0069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba0101fe0 0x7f7ba019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 0x7f7ba019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010cb10 con 0x7f7ba01029a0 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010c990 con 0x7f7ba0101fe0 2024-04-25T13:02:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba7431640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010cc90 con 0x7f7ba0100e90 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba59ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 0x7f7ba019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba59ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 0x7f7ba019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37586/0 (socket says 172.21.15.38:37586) 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba59ce640 1 -- 172.21.15.38:0/2127933754 learned_addr learned my addr 172.21.15.38:0/2127933754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.552+0000 7f7ba49cc640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba0101fe0 0x7f7ba019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba59ce640 1 -- 172.21.15.38:0/2127933754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 msgr2=0x7f7ba0069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba51cd640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 0x7f7ba0069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba59ce640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 0x7f7ba0069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba59ce640 1 -- 172.21.15.38:0/2127933754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba0101fe0 msgr2=0x7f7ba019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba59ce640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba0101fe0 0x7f7ba019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba59ce640 1 -- 172.21.15.38:0/2127933754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba019fe80 con 0x7f7ba01029a0 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba49cc640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba0101fe0 0x7f7ba019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:51.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba51cd640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 0x7f7ba0069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba59ce640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 0x7f7ba019f5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800eb10 tx=0x7f7b9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98004b20 con 0x7f7ba01029a0 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7b98004cc0 con 0x7f7ba01029a0 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba01a0170 con 0x7f7ba01029a0 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98010720 con 0x7f7ba01029a0 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.553+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01a0640 con 0x7f7ba01029a0 2024-04-25T13:02:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.554+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba01081f0 con 0x7f7ba01029a0 2024-04-25T13:02:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.556+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b980108c0 con 0x7f7ba01029a0 2024-04-25T13:02:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.556+0000 7f7b967fc640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7b8c0a3bb0 0x7f7b8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.556+0000 7f7ba51cd640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7b8c0a3bb0 0x7f7b8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.556+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7b980d0ae0 con 0x7f7ba01029a0 2024-04-25T13:02:51.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.557+0000 7f7ba51cd640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7b8c0a3bb0 0x7f7b8c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00baa0 tx=0x7f7b9c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.560+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b98053ff0 con 0x7f7ba01029a0 2024-04-25T13:02:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.778+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7ba0062390 con 0x7f7b8c0a3bb0 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.782+0000 7f7b967fc640 1 -- 172.21.15.38:0/2127933754 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7ba0062390 con 0x7f7b8c0a3bb0 2024-04-25T13:02:51.780 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_1/f2b81406-7843-4642-928a-a1fc987285dd 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7b8c0a3bb0 msgr2=0x7f7b8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7b8c0a3bb0 0x7f7b8c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00baa0 tx=0x7f7b9c005d90 comp rx=0 tx=0).stop 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 msgr2=0x7f7ba019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 0x7f7ba019f5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800eb10 tx=0x7f7b9800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 shutdown_connections 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f7b8c0a3bb0 0x7f7b8c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ba01029a0 0x7f7ba019f5c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba0101fe0 0x7f7ba019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 --2- 172.21.15.38:0/2127933754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ba0100e90 0x7f7ba0069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 >> 172.21.15.38:0/2127933754 conn(0x7f7ba00fc400 msgr2=0x7f7ba01053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 shutdown_connections 2024-04-25T13:02:51.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.784+0000 7f7ba7431640 1 -- 172.21.15.38:0/2127933754 wait complete. 2024-04-25T13:02:51.793 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 -- 172.21.15.38:0/1781667077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8101fa0 msgr2=0x7f8fc81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/1781667077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8101fa0 0x7f8fc81023c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009860 tx=0x7f8fbc02a340 comp rx=0 tx=0).stop 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 -- 172.21.15.38:0/1781667077 shutdown_connections 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/1781667077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103b50 0x7f8fc810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/1781667077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc8103190 0x7f8fc8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/1781667077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8101fa0 0x7f8fc81023c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 -- 172.21.15.38:0/1781667077 >> 172.21.15.38:0/1781667077 conn(0x7f8fc80fd530 msgr2=0x7f8fc80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 -- 172.21.15.38:0/1781667077 shutdown_connections 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.898+0000 7f8fcf20b640 1 -- 172.21.15.38:0/1781667077 wait complete. 2024-04-25T13:02:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 Processor -- start 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 -- start start 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc8101fa0 0x7f8fc8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 0x7f8fc8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8103b50 0x7f8fc8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810dc00 con 0x7f8fc8103190 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810da80 con 0x7f8fc8103b50 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcf20b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810dd80 con 0x7f8fc8101fa0 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 0x7f8fc8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 0x7f8fc8074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37598/0 (socket says 172.21.15.38:37598) 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fc7fff640 1 -- 172.21.15.38:0/439836554 learned_addr learned my addr 172.21.15.38:0/439836554 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fcd7a8640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8103b50 0x7f8fc8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.899+0000 7f8fccfa7640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc8101fa0 0x7f8fc8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc7fff640 1 -- 172.21.15.38:0/439836554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc8101fa0 msgr2=0x7f8fc8075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc7fff640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc8101fa0 0x7f8fc8075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc7fff640 1 -- 172.21.15.38:0/439836554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8103b50 msgr2=0x7f8fc8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc7fff640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8103b50 0x7f8fc8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc7fff640 1 -- 172.21.15.38:0/439836554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc8075110 con 0x7f8fc8103190 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fcd7a8640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8103b50 0x7f8fc8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc7fff640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 0x7f8fc8074340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400d770 tx=0x7f8fb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4017070 con 0x7f8fc8103190 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8fb40054f0 con 0x7f8fc8103190 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc81a23e0 con 0x7f8fc8103190 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4004de0 con 0x7f8fc8103190 2024-04-25T13:02:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.900+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc81a28e0 con 0x7f8fc8103190 2024-04-25T13:02:51.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.901+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc81092c0 con 0x7f8fc8103190 2024-04-25T13:02:51.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.902+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fb4005020 con 0x7f8fc8103190 2024-04-25T13:02:51.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.903+0000 7f8fc5ffb640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fb80a3bb0 0x7f8fb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:51.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.903+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8fb40cfcf0 con 0x7f8fc8103190 2024-04-25T13:02:51.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.903+0000 7f8fccfa7640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fb80a3bb0 0x7f8fb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:51.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.903+0000 7f8fccfa7640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fb80a3bb0 0x7f8fb80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009830 tx=0x7f8fbc030540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:51.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.908+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fb40d4070 con 0x7f8fc8103190 2024-04-25T13:02:51.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:51.973+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8fb4098840 con 0x7f8fc8103190 2024-04-25T13:02:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:51 smithi038 ceph-mon[29400]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:51 smithi038 ceph-mon[29400]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:51 smithi038 ceph-mon[29400]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:51 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] up:active 2024-04-25T13:02:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:51 smithi038 ceph-mon[29400]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:02:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:51 smithi038 ceph-mon[29400]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:02:52.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.140+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fc802d810 con 0x7f8fc8103190 2024-04-25T13:02:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.141+0000 7f8fc5ffb640 1 -- 172.21.15.38:0/439836554 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11938 (secure 0 0 0) 0x7f8fb409c8d0 con 0x7f8fc8103190 2024-04-25T13:02:52.139 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:52.139 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:02:51.699933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24484,24518,24482,24466,14625]},"id":1}]} 2024-04-25T13:02:52.140 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fb80a3bb0 msgr2=0x7f8fb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fb80a3bb0 0x7f8fb80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009830 tx=0x7f8fbc030540 comp rx=0 tx=0).stop 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 msgr2=0x7f8fc8074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 0x7f8fc8074340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400d770 tx=0x7f8fb400dc30 comp rx=0 tx=0).stop 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 shutdown_connections 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8fb80a3bb0 0x7f8fb80a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fc8103b50 0x7f8fc8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc8103190 0x7f8fc8074340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 --2- 172.21.15.38:0/439836554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc8101fa0 0x7f8fc8075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 >> 172.21.15.38:0/439836554 conn(0x7f8fc80fd530 msgr2=0x7f8fc8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:52.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 shutdown_connections 2024-04-25T13:02:52.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.143+0000 7f8fcf20b640 1 -- 172.21.15.38:0/439836554 wait complete. 2024-04-25T13:02:52.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:02:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:51 smithi178 ceph-mon[33233]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:51 smithi178 ceph-mon[33233]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:51 smithi178 ceph-mon[33233]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:51 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] up:active 2024-04-25T13:02:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:51 smithi178 ceph-mon[33233]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:02:52.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:51 smithi178 ceph-mon[33233]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 -- 172.21.15.38:0/4278850188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 msgr2=0x7f13781023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 --2- 172.21.15.38:0/4278850188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f13781023c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1368009ca0 tx=0x7f136802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 -- 172.21.15.38:0/4278850188 shutdown_connections 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 --2- 172.21.15.38:0/4278850188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1378103b50 0x7f137810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 --2- 172.21.15.38:0/4278850188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1378103190 0x7f1378103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 --2- 172.21.15.38:0/4278850188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f13781023c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.257+0000 7f137e059640 1 -- 172.21.15.38:0/4278850188 >> 172.21.15.38:0/4278850188 conn(0x7f13780fd530 msgr2=0x7f13780ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 -- 172.21.15.38:0/4278850188 shutdown_connections 2024-04-25T13:02:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 -- 172.21.15.38:0/4278850188 wait complete. 2024-04-25T13:02:52.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 Processor -- start 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 -- start start 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f1378075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1378103190 0x7f1378074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1378103b50 0x7f1378074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.258+0000 7f137e059640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f137810dc50 con 0x7f1378101fa0 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f137e059640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f137810dad0 con 0x7f1378103b50 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f137e059640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f137810ddd0 con 0x7f1378103190 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f1378075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f1378075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37616/0 (socket says 172.21.15.38:37616) 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 -- 172.21.15.38:0/3947790792 learned_addr learned my addr 172.21.15.38:0/3947790792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f1376ffd640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1378103190 0x7f1378074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f1377fff640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1378103b50 0x7f1378074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 -- 172.21.15.38:0/3947790792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1378103190 msgr2=0x7f1378074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1378103190 0x7f1378074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 -- 172.21.15.38:0/3947790792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1378103b50 msgr2=0x7f1378074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1378103b50 0x7f1378074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 -- 172.21.15.38:0/3947790792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1378075110 con 0x7f1378101fa0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f13777fe640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f1378075d00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1368005e30 tx=0x7f136802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368003bf0 con 0x7f1378101fa0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1368003d90 con 0x7f1378101fa0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13781a2410 con 0x7f1378101fa0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136802fcf0 con 0x7f1378101fa0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.259+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781a2960 con 0x7f1378101fa0 2024-04-25T13:02:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.260+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13781092c0 con 0x7f1378101fa0 2024-04-25T13:02:52.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.261+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1368004040 con 0x7f1378101fa0 2024-04-25T13:02:52.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.262+0000 7f1374ff9640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f13600a3c40 0x7f13600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.262+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f13680f1930 con 0x7f1378101fa0 2024-04-25T13:02:52.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.262+0000 7f1376ffd640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f13600a3c40 0x7f13600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.262+0000 7f1376ffd640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f13600a3c40 0x7f13600a6100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1378075a70 tx=0x7f136c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.266+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f136802fe90 con 0x7f1378101fa0 2024-04-25T13:02:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:51 smithi104 ceph-mon[32726]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:51 smithi104 ceph-mon[32726]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:51 smithi104 ceph-mon[32726]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:52.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:51 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] up:active 2024-04-25T13:02:52.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:51 smithi104 ceph-mon[32726]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:02:52.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:51 smithi104 ceph-mon[32726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:02:52.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.455+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1378062b50 con 0x7f1378101fa0 2024-04-25T13:02:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.456+0000 7f1374ff9640 1 -- 172.21.15.38:0/3947790792 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22172 (secure 0 0 0) 0x7f13680be470 con 0x7f1378101fa0 2024-04-25T13:02:52.455 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:52.455 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:02:45.729306+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:02:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f13600a3c40 msgr2=0x7f13600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f13600a3c40 0x7f13600a6100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1378075a70 tx=0x7f136c00b390 comp rx=0 tx=0).stop 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 msgr2=0x7f1378075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f1378075d00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1368005e30 tx=0x7f136802f3a0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 shutdown_connections 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f13600a3c40 0x7f13600a6100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1378103b50 0x7f1378074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1378103190 0x7f1378074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 --2- 172.21.15.38:0/3947790792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1378101fa0 0x7f1378075d00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 >> 172.21.15.38:0/3947790792 conn(0x7f13780fd530 msgr2=0x7f1378106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 shutdown_connections 2024-04-25T13:02:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.458+0000 7f137e059640 1 -- 172.21.15.38:0/3947790792 wait complete. 2024-04-25T13:02:52.468 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:02:52.468 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:02:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 -- 172.21.15.38:0/730342637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324102040 msgr2=0x7f73241024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 --2- 172.21.15.38:0/730342637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324102040 0x7f73241024c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7314009900 tx=0x7f731402a410 comp rx=0 tx=0).stop 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 -- 172.21.15.38:0/730342637 shutdown_connections 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 --2- 172.21.15.38:0/730342637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7324102a00 0x7f7324109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 --2- 172.21.15.38:0/730342637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324102040 0x7f73241024c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 --2- 172.21.15.38:0/730342637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324100e60 0x7f7324101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 -- 172.21.15.38:0/730342637 >> 172.21.15.38:0/730342637 conn(0x7f73240fc440 msgr2=0x7f73240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 -- 172.21.15.38:0/730342637 shutdown_connections 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 -- 172.21.15.38:0/730342637 wait complete. 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.572+0000 7f732b9e1640 1 Processor -- start 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 -- start start 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 0x7f7324104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7324102040 0x7f73241034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324102a00 0x7f73241039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f732410cb50 con 0x7f7324100e60 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f732410c9d0 con 0x7f7324102040 2024-04-25T13:02:52.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f732410ccd0 con 0x7f7324102a00 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 0x7f7324104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 0x7f7324104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37632/0 (socket says 172.21.15.38:37632) 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 -- 172.21.15.38:0/2119785360 learned_addr learned my addr 172.21.15.38:0/2119785360 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f7328f7c640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7324102040 0x7f73241034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f7329f7e640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324102a00 0x7f73241039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 -- 172.21.15.38:0/2119785360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324102a00 msgr2=0x7f73241039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324102a00 0x7f73241039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 -- 172.21.15.38:0/2119785360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7324102040 msgr2=0x7f73241034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7324102040 0x7f73241034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 -- 172.21.15.38:0/2119785360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7324104280 con 0x7f7324100e60 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f7329f7e640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324102a00 0x7f73241039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732977d640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 0x7f7324104e70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f732000eaa0 tx=0x7f732000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:52.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320016070 con 0x7f7324100e60 2024-04-25T13:02:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7320003e40 con 0x7f7324100e60 2024-04-25T13:02:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241a2440 con 0x7f7324100e60 2024-04-25T13:02:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.573+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200116d0 con 0x7f7324100e60 2024-04-25T13:02:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.574+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a2960 con 0x7f7324100e60 2024-04-25T13:02:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.574+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73240691c0 con 0x7f7324100e60 2024-04-25T13:02:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.575+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7320019070 con 0x7f7324100e60 2024-04-25T13:02:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.576+0000 7f731a7fc640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f73100a3cc0 0x7f73100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.576+0000 7f7328f7c640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f73100a3cc0 0x7f73100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.576+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f73200d04a0 con 0x7f7324100e60 2024-04-25T13:02:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.577+0000 7f7328f7c640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f73100a3cc0 0x7f73100a6180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7314000c00 tx=0x7f7314031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:52.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.581+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f732009ced0 con 0x7f7324100e60 2024-04-25T13:02:52.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.768+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73240fe5e0 con 0x7f73100a3cc0 2024-04-25T13:02:52.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.775+0000 7f731a7fc640 1 -- 172.21.15.38:0/2119785360 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258879 (secure 0 0 0) 0x7f73240fe5e0 con 0x7f73100a3cc0 2024-04-25T13:02:52.777 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:52.778 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f73100a3cc0 msgr2=0x7f73100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f73100a3cc0 0x7f73100a6180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7314000c00 tx=0x7f7314031510 comp rx=0 tx=0).stop 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 msgr2=0x7f7324104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 0x7f7324104e70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f732000eaa0 tx=0x7f732000ef60 comp rx=0 tx=0).stop 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 shutdown_connections 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f73100a3cc0 0x7f73100a6180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7324102a00 0x7f73241039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7324102040 0x7f73241034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 --2- 172.21.15.38:0/2119785360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7324100e60 0x7f7324104e70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 >> 172.21.15.38:0/2119785360 conn(0x7f73240fc440 msgr2=0x7f73240fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 shutdown_connections 2024-04-25T13:02:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.778+0000 7f732b9e1640 1 -- 172.21.15.38:0/2119785360 wait complete. 2024-04-25T13:02:52.788 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-25T13:02:51.282414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594436,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":143,"num_read_kb":126,"num_write":309,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372836,"kb_used_data":11148,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568988,"statfs":{"total":1151940427776,"available":1151558643712,"internally_reserved":0,"allocated":11415552,"data_stored":7082544,"data_compressed":109923,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4068,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":128,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.391233"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410619+0000","last_change":"2024-04-25T13:02:44.736489+0000","last_active":"2024-04-25T13:02:46.410619+0000","last_peered":"2024-04-25T13:02:46.410619+0000","last_clean":"2024-04-25T13:02:46.410619+0000","last_became_active":"2024-04-25T13:02:44.735677+0000","last_became_peered":"2024-04-25T13:02:44.735677+0000","last_unstale":"2024-04-25T13:02:46.410619+0000","last_undegraded":"2024-04-25T13:02:46.410619+0000","last_fullsized":"2024-04-25T13:02:46.410619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:59.810878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284870+0000","last_change":"2024-04-25T13:02:44.740349+0000","last_active":"2024-04-25T13:02:46.284870+0000","last_peered":"2024-04-25T13:02:46.284870+0000","last_clean":"2024-04-25T13:02:46.284870+0000","last_became_active":"2024-04-25T13:02:44.740024+0000","last_became_peered":"2024-04-25T13:02:44.740024+0000","last_unstale":"2024-04-25T13:02:46.284870+0000","last_undegraded":"2024-04-25T13:02:46.284870+0000","last_fullsized":"2024-04-25T13:02:46.284870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:35.763278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284734+0000","last_change":"2024-04-25T13:02:44.742136+0000","last_active":"2024-04-25T13:02:46.284734+0000","last_peered":"2024-04-25T13:02:46.284734+0000","last_clean":"2024-04-25T13:02:46.284734+0000","last_became_active":"2024-04-25T13:02:44.741840+0000","last_became_peered":"2024-04-25T13:02:44.741840+0000","last_unstale":"2024-04-25T13:02:46.284734+0000","last_undegraded":"2024-04-25T13:02:46.284734+0000","last_fullsized":"2024-04-25T13:02:46.284734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:48.690435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410346+0000","last_change":"2024-04-25T13:02:44.733349+0000","last_active":"2024-04-25T13:02:46.410346+0000","last_peered":"2024-04-25T13:02:46.410346+0000","last_clean":"2024-04-25T13:02:46.410346+0000","last_became_active":"2024-04-25T13:02:44.732389+0000","last_became_peered":"2024-04-25T13:02:44.732389+0000","last_unstale":"2024-04-25T13:02:46.410346+0000","last_undegraded":"2024-04-25T13:02:46.410346+0000","last_fullsized":"2024-04-25T13:02:46.410346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:28.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.896983+0000","last_change":"2024-04-25T13:02:44.732852+0000","last_active":"2024-04-25T13:02:45.896983+0000","last_peered":"2024-04-25T13:02:45.896983+0000","last_clean":"2024-04-25T13:02:45.896983+0000","last_became_active":"2024-04-25T13:02:44.732536+0000","last_became_peered":"2024-04-25T13:02:44.732536+0000","last_unstale":"2024-04-25T13:02:45.896983+0000","last_undegraded":"2024-04-25T13:02:45.896983+0000","last_fullsized":"2024-04-25T13:02:45.896983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:58.358684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036405+0000","last_change":"2024-04-25T13:02:44.731636+0000","last_active":"2024-04-25T13:02:46.036405+0000","last_peered":"2024-04-25T13:02:46.036405+0000","last_clean":"2024-04-25T13:02:46.036405+0000","last_became_active":"2024-04-25T13:02:44.731334+0000","last_became_peered":"2024-04-25T13:02:44.731334+0000","last_unstale":"2024-04-25T13:02:46.036405+0000","last_undegraded":"2024-04-25T13:02:46.036405+0000","last_fullsized":"2024-04-25T13:02:46.036405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:00.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410389+0000","last_change":"2024-04-25T13:02:44.737275+0000","last_active":"2024-04-25T13:02:46.410389+0000","last_peered":"2024-04-25T13:02:46.410389+0000","last_clean":"2024-04-25T13:02:46.410389+0000","last_became_active":"2024-04-25T13:02:44.736924+0000","last_became_peered":"2024-04-25T13:02:44.736924+0000","last_unstale":"2024-04-25T13:02:46.410389+0000","last_undegraded":"2024-04-25T13:02:46.410389+0000","last_fullsized":"2024-04-25T13:02:46.410389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.564465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732223+0000","last_change":"2024-04-25T13:02:44.736736+0000","last_active":"2024-04-25T13:02:45.732223+0000","last_peered":"2024-04-25T13:02:45.732223+0000","last_clean":"2024-04-25T13:02:45.732223+0000","last_became_active":"2024-04-25T13:02:44.736403+0000","last_became_peered":"2024-04-25T13:02:44.736403+0000","last_unstale":"2024-04-25T13:02:45.732223+0000","last_undegraded":"2024-04-25T13:02:45.732223+0000","last_fullsized":"2024-04-25T13:02:45.732223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:54.531475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733075+0000","last_change":"2024-04-25T13:02:44.738124+0000","last_active":"2024-04-25T13:02:45.733075+0000","last_peered":"2024-04-25T13:02:45.733075+0000","last_clean":"2024-04-25T13:02:45.733075+0000","last_became_active":"2024-04-25T13:02:44.737572+0000","last_became_peered":"2024-04-25T13:02:44.737572+0000","last_unstale":"2024-04-25T13:02:45.733075+0000","last_undegraded":"2024-04-25T13:02:45.733075+0000","last_fullsized":"2024-04-25T13:02:45.733075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:00.692786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036657+0000","last_change":"2024-04-25T13:02:44.730108+0000","last_active":"2024-04-25T13:02:46.036657+0000","last_peered":"2024-04-25T13:02:46.036657+0000","last_clean":"2024-04-25T13:02:46.036657+0000","last_became_active":"2024-04-25T13:02:44.729449+0000","last_became_peered":"2024-04-25T13:02:44.729449+0000","last_unstale":"2024-04-25T13:02:46.036657+0000","last_undegraded":"2024-04-25T13:02:46.036657+0000","last_fullsized":"2024-04-25T13:02:46.036657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:19.888190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.411032+0000","last_change":"2024-04-25T13:02:44.730902+0000","last_active":"2024-04-25T13:02:46.411032+0000","last_peered":"2024-04-25T13:02:46.411032+0000","last_clean":"2024-04-25T13:02:46.411032+0000","last_became_active":"2024-04-25T13:02:44.730560+0000","last_became_peered":"2024-04-25T13:02:44.730560+0000","last_unstale":"2024-04-25T13:02:46.411032+0000","last_undegraded":"2024-04-25T13:02:46.411032+0000","last_fullsized":"2024-04-25T13:02:46.411032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:07.133221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410371+0000","last_change":"2024-04-25T13:02:44.730966+0000","last_active":"2024-04-25T13:02:46.410371+0000","last_peered":"2024-04-25T13:02:46.410371+0000","last_clean":"2024-04-25T13:02:46.410371+0000","last_became_active":"2024-04-25T13:02:44.730546+0000","last_became_peered":"2024-04-25T13:02:44.730546+0000","last_unstale":"2024-04-25T13:02:46.410371+0000","last_undegraded":"2024-04-25T13:02:46.410371+0000","last_fullsized":"2024-04-25T13:02:46.410371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.188491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410202+0000","last_change":"2024-04-25T13:02:44.739301+0000","last_active":"2024-04-25T13:02:46.410202+0000","last_peered":"2024-04-25T13:02:46.410202+0000","last_clean":"2024-04-25T13:02:46.410202+0000","last_became_active":"2024-04-25T13:02:44.738704+0000","last_became_peered":"2024-04-25T13:02:44.738704+0000","last_unstale":"2024-04-25T13:02:46.410202+0000","last_undegraded":"2024-04-25T13:02:46.410202+0000","last_fullsized":"2024-04-25T13:02:46.410202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:44.373522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284491+0000","last_change":"2024-04-25T13:02:44.738703+0000","last_active":"2024-04-25T13:02:46.284491+0000","last_peered":"2024-04-25T13:02:46.284491+0000","last_clean":"2024-04-25T13:02:46.284491+0000","last_became_active":"2024-04-25T13:02:44.738373+0000","last_became_peered":"2024-04-25T13:02:44.738373+0000","last_unstale":"2024-04-25T13:02:46.284491+0000","last_undegraded":"2024-04-25T13:02:46.284491+0000","last_fullsized":"2024-04-25T13:02:46.284491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:29.106615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284536+0000","last_change":"2024-04-25T13:02:44.743336+0000","last_active":"2024-04-25T13:02:46.284536+0000","last_peered":"2024-04-25T13:02:46.284536+0000","last_clean":"2024-04-25T13:02:46.284536+0000","last_became_active":"2024-04-25T13:02:44.743032+0000","last_became_peered":"2024-04-25T13:02:44.743032+0000","last_unstale":"2024-04-25T13:02:46.284536+0000","last_undegraded":"2024-04-25T13:02:46.284536+0000","last_fullsized":"2024-04-25T13:02:46.284536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:18.403138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732563+0000","last_change":"2024-04-25T13:02:44.739403+0000","last_active":"2024-04-25T13:02:45.732563+0000","last_peered":"2024-04-25T13:02:45.732563+0000","last_clean":"2024-04-25T13:02:45.732563+0000","last_became_active":"2024-04-25T13:02:44.739097+0000","last_became_peered":"2024-04-25T13:02:44.739097+0000","last_unstale":"2024-04-25T13:02:45.732563+0000","last_undegraded":"2024-04-25T13:02:45.732563+0000","last_fullsized":"2024-04-25T13:02:45.732563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:30.519343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410693+0000","last_change":"2024-04-25T13:02:44.736821+0000","last_active":"2024-04-25T13:02:46.410693+0000","last_peered":"2024-04-25T13:02:46.410693+0000","last_clean":"2024-04-25T13:02:46.410693+0000","last_became_active":"2024-04-25T13:02:44.736099+0000","last_became_peered":"2024-04-25T13:02:44.736099+0000","last_unstale":"2024-04-25T13:02:46.410693+0000","last_undegraded":"2024-04-25T13:02:46.410693+0000","last_fullsized":"2024-04-25T13:02:46.410693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:54.858845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897609+0000","last_change":"2024-04-25T13:02:44.731035+0000","last_active":"2024-04-25T13:02:45.897609+0000","last_peered":"2024-04-25T13:02:45.897609+0000","last_clean":"2024-04-25T13:02:45.897609+0000","last_became_active":"2024-04-25T13:02:44.730717+0000","last_became_peered":"2024-04-25T13:02:44.730717+0000","last_unstale":"2024-04-25T13:02:45.897609+0000","last_undegraded":"2024-04-25T13:02:45.897609+0000","last_fullsized":"2024-04-25T13:02:45.897609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:09.061183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410231+0000","last_change":"2024-04-25T13:02:44.735851+0000","last_active":"2024-04-25T13:02:46.410231+0000","last_peered":"2024-04-25T13:02:46.410231+0000","last_clean":"2024-04-25T13:02:46.410231+0000","last_became_active":"2024-04-25T13:02:44.735559+0000","last_became_peered":"2024-04-25T13:02:44.735559+0000","last_unstale":"2024-04-25T13:02:46.410231+0000","last_undegraded":"2024-04-25T13:02:46.410231+0000","last_fullsized":"2024-04-25T13:02:46.410231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.952094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.951474+0000","last_change":"2024-04-25T13:02:43.909800+0000","last_active":"2024-04-25T13:02:48.951474+0000","last_peered":"2024-04-25T13:02:48.951474+0000","last_clean":"2024-04-25T13:02:48.951474+0000","last_became_active":"2024-04-25T13:02:43.909463+0000","last_became_peered":"2024-04-25T13:02:43.909463+0000","last_unstale":"2024-04-25T13:02:48.951474+0000","last_undegraded":"2024-04-25T13:02:48.951474+0000","last_fullsized":"2024-04-25T13:02:48.951474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:31.495586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732445+0000","last_change":"2024-04-25T13:02:44.736710+0000","last_active":"2024-04-25T13:02:45.732445+0000","last_peered":"2024-04-25T13:02:45.732445+0000","last_clean":"2024-04-25T13:02:45.732445+0000","last_became_active":"2024-04-25T13:02:44.736385+0000","last_became_peered":"2024-04-25T13:02:44.736385+0000","last_unstale":"2024-04-25T13:02:45.732445+0000","last_undegraded":"2024-04-25T13:02:45.732445+0000","last_fullsized":"2024-04-25T13:02:45.732445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.216946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732958+0000","last_change":"2024-04-25T13:02:43.918766+0000","last_active":"2024-04-25T13:02:45.732958+0000","last_peered":"2024-04-25T13:02:45.732958+0000","last_clean":"2024-04-25T13:02:45.732958+0000","last_became_active":"2024-04-25T13:02:43.918283+0000","last_became_peered":"2024-04-25T13:02:43.918283+0000","last_unstale":"2024-04-25T13:02:45.732958+0000","last_undegraded":"2024-04-25T13:02:45.732958+0000","last_fullsized":"2024-04-25T13:02:45.732958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:17.986777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284756+0000","last_change":"2024-04-25T13:02:44.735806+0000","last_active":"2024-04-25T13:02:46.284756+0000","last_peered":"2024-04-25T13:02:46.284756+0000","last_clean":"2024-04-25T13:02:46.284756+0000","last_became_active":"2024-04-25T13:02:44.735507+0000","last_became_peered":"2024-04-25T13:02:44.735507+0000","last_unstale":"2024-04-25T13:02:46.284756+0000","last_undegraded":"2024-04-25T13:02:46.284756+0000","last_fullsized":"2024-04-25T13:02:46.284756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:22.230015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.702593+0000","last_change":"2024-04-25T13:02:43.910055+0000","last_active":"2024-04-25T13:02:50.702593+0000","last_peered":"2024-04-25T13:02:50.702593+0000","last_clean":"2024-04-25T13:02:50.702593+0000","last_became_active":"2024-04-25T13:02:43.909762+0000","last_became_peered":"2024-04-25T13:02:43.909762+0000","last_unstale":"2024-04-25T13:02:50.702593+0000","last_undegraded":"2024-04-25T13:02:50.702593+0000","last_fullsized":"2024-04-25T13:02:50.702593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:47.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732985+0000","last_change":"2024-04-25T13:02:44.733253+0000","last_active":"2024-04-25T13:02:45.732985+0000","last_peered":"2024-04-25T13:02:45.732985+0000","last_clean":"2024-04-25T13:02:45.732985+0000","last_became_active":"2024-04-25T13:02:44.732917+0000","last_became_peered":"2024-04-25T13:02:44.732917+0000","last_unstale":"2024-04-25T13:02:45.732985+0000","last_undegraded":"2024-04-25T13:02:45.732985+0000","last_fullsized":"2024-04-25T13:02:45.732985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:10.380810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.700941+0000","last_change":"2024-04-25T13:02:43.901490+0000","last_active":"2024-04-25T13:02:50.700941+0000","last_peered":"2024-04-25T13:02:50.700941+0000","last_clean":"2024-04-25T13:02:50.700941+0000","last_became_active":"2024-04-25T13:02:43.900807+0000","last_became_peered":"2024-04-25T13:02:43.900807+0000","last_unstale":"2024-04-25T13:02:50.700941+0000","last_undegraded":"2024-04-25T13:02:50.700941+0000","last_fullsized":"2024-04-25T13:02:50.700941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.101575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733106+0000","last_change":"2024-04-25T13:02:44.744380+0000","last_active":"2024-04-25T13:02:45.733106+0000","last_peered":"2024-04-25T13:02:45.733106+0000","last_clean":"2024-04-25T13:02:45.733106+0000","last_became_active":"2024-04-25T13:02:44.744098+0000","last_became_peered":"2024-04-25T13:02:44.744098+0000","last_unstale":"2024-04-25T13:02:45.733106+0000","last_undegraded":"2024-04-25T13:02:45.733106+0000","last_fullsized":"2024-04-25T13:02:45.733106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:53.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732844+0000","last_change":"2024-04-25T13:02:43.908588+0000","last_active":"2024-04-25T13:02:45.732844+0000","last_peered":"2024-04-25T13:02:45.732844+0000","last_clean":"2024-04-25T13:02:45.732844+0000","last_became_active":"2024-04-25T13:02:43.908219+0000","last_became_peered":"2024-04-25T13:02:43.908219+0000","last_unstale":"2024-04-25T13:02:45.732844+0000","last_undegraded":"2024-04-25T13:02:45.732844+0000","last_fullsized":"2024-04-25T13:02:45.732844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:44.126509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897077+0000","last_change":"2024-04-25T13:02:44.737072+0000","last_active":"2024-04-25T13:02:45.897077+0000","last_peered":"2024-04-25T13:02:45.897077+0000","last_clean":"2024-04-25T13:02:45.897077+0000","last_became_active":"2024-04-25T13:02:44.736750+0000","last_became_peered":"2024-04-25T13:02:44.736750+0000","last_unstale":"2024-04-25T13:02:45.897077+0000","last_undegraded":"2024-04-25T13:02:45.897077+0000","last_fullsized":"2024-04-25T13:02:45.897077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:42.933740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950105+0000","last_change":"2024-04-25T13:02:43.913516+0000","last_active":"2024-04-25T13:02:48.950105+0000","last_peered":"2024-04-25T13:02:48.950105+0000","last_clean":"2024-04-25T13:02:48.950105+0000","last_became_active":"2024-04-25T13:02:43.913067+0000","last_became_peered":"2024-04-25T13:02:43.913067+0000","last_unstale":"2024-04-25T13:02:48.950105+0000","last_undegraded":"2024-04-25T13:02:48.950105+0000","last_fullsized":"2024-04-25T13:02:48.950105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:55.004598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732957+0000","last_change":"2024-04-25T13:02:44.741433+0000","last_active":"2024-04-25T13:02:45.732957+0000","last_peered":"2024-04-25T13:02:45.732957+0000","last_clean":"2024-04-25T13:02:45.732957+0000","last_became_active":"2024-04-25T13:02:44.741114+0000","last_became_peered":"2024-04-25T13:02:44.741114+0000","last_unstale":"2024-04-25T13:02:45.732957+0000","last_undegraded":"2024-04-25T13:02:45.732957+0000","last_fullsized":"2024-04-25T13:02:45.732957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:02.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.951955+0000","last_change":"2024-04-25T13:02:43.900699+0000","last_active":"2024-04-25T13:02:48.951955+0000","last_peered":"2024-04-25T13:02:48.951955+0000","last_clean":"2024-04-25T13:02:48.951955+0000","last_became_active":"2024-04-25T13:02:43.900306+0000","last_became_peered":"2024-04-25T13:02:43.900306+0000","last_unstale":"2024-04-25T13:02:48.951955+0000","last_undegraded":"2024-04-25T13:02:48.951955+0000","last_fullsized":"2024-04-25T13:02:48.951955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.720365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732297+0000","last_change":"2024-04-25T13:02:44.733157+0000","last_active":"2024-04-25T13:02:45.732297+0000","last_peered":"2024-04-25T13:02:45.732297+0000","last_clean":"2024-04-25T13:02:45.732297+0000","last_became_active":"2024-04-25T13:02:44.732836+0000","last_became_peered":"2024-04-25T13:02:44.732836+0000","last_unstale":"2024-04-25T13:02:45.732297+0000","last_undegraded":"2024-04-25T13:02:45.732297+0000","last_fullsized":"2024-04-25T13:02:45.732297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:30.189708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732615+0000","last_change":"2024-04-25T13:02:43.913362+0000","last_active":"2024-04-25T13:02:45.732615+0000","last_peered":"2024-04-25T13:02:45.732615+0000","last_clean":"2024-04-25T13:02:45.732615+0000","last_became_active":"2024-04-25T13:02:43.912763+0000","last_became_peered":"2024-04-25T13:02:43.912763+0000","last_unstale":"2024-04-25T13:02:45.732615+0000","last_undegraded":"2024-04-25T13:02:45.732615+0000","last_fullsized":"2024-04-25T13:02:45.732615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:12.216165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897705+0000","last_change":"2024-04-25T13:02:44.732662+0000","last_active":"2024-04-25T13:02:45.897705+0000","last_peered":"2024-04-25T13:02:45.897705+0000","last_clean":"2024-04-25T13:02:45.897705+0000","last_became_active":"2024-04-25T13:02:44.732319+0000","last_became_peered":"2024-04-25T13:02:44.732319+0000","last_unstale":"2024-04-25T13:02:45.897705+0000","last_undegraded":"2024-04-25T13:02:45.897705+0000","last_fullsized":"2024-04-25T13:02:45.897705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:51.683916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.938932+0000","last_change":"2024-04-25T13:02:43.911019+0000","last_active":"2024-04-25T13:02:47.938932+0000","last_peered":"2024-04-25T13:02:47.938932+0000","last_clean":"2024-04-25T13:02:47.938932+0000","last_became_active":"2024-04-25T13:02:43.910707+0000","last_became_peered":"2024-04-25T13:02:43.910707+0000","last_unstale":"2024-04-25T13:02:47.938932+0000","last_undegraded":"2024-04-25T13:02:47.938932+0000","last_fullsized":"2024-04-25T13:02:47.938932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:07.042580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733012+0000","last_change":"2024-04-25T13:02:44.732791+0000","last_active":"2024-04-25T13:02:45.733012+0000","last_peered":"2024-04-25T13:02:45.733012+0000","last_clean":"2024-04-25T13:02:45.733012+0000","last_became_active":"2024-04-25T13:02:44.732484+0000","last_became_peered":"2024-04-25T13:02:44.732484+0000","last_unstale":"2024-04-25T13:02:45.733012+0000","last_undegraded":"2024-04-25T13:02:45.733012+0000","last_fullsized":"2024-04-25T13:02:45.733012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:36.164774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.937394+0000","last_change":"2024-04-25T13:02:43.912447+0000","last_active":"2024-04-25T13:02:47.937394+0000","last_peered":"2024-04-25T13:02:47.937394+0000","last_clean":"2024-04-25T13:02:47.937394+0000","last_became_active":"2024-04-25T13:02:43.912091+0000","last_became_peered":"2024-04-25T13:02:43.912091+0000","last_unstale":"2024-04-25T13:02:47.937394+0000","last_undegraded":"2024-04-25T13:02:47.937394+0000","last_fullsized":"2024-04-25T13:02:47.937394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:41.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284787+0000","last_change":"2024-04-25T13:02:44.739021+0000","last_active":"2024-04-25T13:02:46.284787+0000","last_peered":"2024-04-25T13:02:46.284787+0000","last_clean":"2024-04-25T13:02:46.284787+0000","last_became_active":"2024-04-25T13:02:44.737945+0000","last_became_peered":"2024-04-25T13:02:44.737945+0000","last_unstale":"2024-04-25T13:02:46.284787+0000","last_undegraded":"2024-04-25T13:02:46.284787+0000","last_fullsized":"2024-04-25T13:02:46.284787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:09.405021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733106+0000","last_change":"2024-04-25T13:02:43.909061+0000","last_active":"2024-04-25T13:02:45.733106+0000","last_peered":"2024-04-25T13:02:45.733106+0000","last_clean":"2024-04-25T13:02:45.733106+0000","last_became_active":"2024-04-25T13:02:43.908739+0000","last_became_peered":"2024-04-25T13:02:43.908739+0000","last_unstale":"2024-04-25T13:02:45.733106+0000","last_undegraded":"2024-04-25T13:02:45.733106+0000","last_fullsized":"2024-04-25T13:02:45.733106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:25.720808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732998+0000","last_change":"2024-04-25T13:02:44.734346+0000","last_active":"2024-04-25T13:02:45.732998+0000","last_peered":"2024-04-25T13:02:45.732998+0000","last_clean":"2024-04-25T13:02:45.732998+0000","last_became_active":"2024-04-25T13:02:44.734045+0000","last_became_peered":"2024-04-25T13:02:44.734045+0000","last_unstale":"2024-04-25T13:02:45.732998+0000","last_undegraded":"2024-04-25T13:02:45.732998+0000","last_fullsized":"2024-04-25T13:02:45.732998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:42.137455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.939198+0000","last_change":"2024-04-25T13:02:43.910212+0000","last_active":"2024-04-25T13:02:47.939198+0000","last_peered":"2024-04-25T13:02:47.939198+0000","last_clean":"2024-04-25T13:02:47.939198+0000","last_became_active":"2024-04-25T13:02:43.909873+0000","last_became_peered":"2024-04-25T13:02:43.909873+0000","last_unstale":"2024-04-25T13:02:47.939198+0000","last_undegraded":"2024-04-25T13:02:47.939198+0000","last_fullsized":"2024-04-25T13:02:47.939198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.716434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897170+0000","last_change":"2024-04-25T13:02:44.731633+0000","last_active":"2024-04-25T13:02:45.897170+0000","last_peered":"2024-04-25T13:02:45.897170+0000","last_clean":"2024-04-25T13:02:45.897170+0000","last_became_active":"2024-04-25T13:02:44.731360+0000","last_became_peered":"2024-04-25T13:02:44.731360+0000","last_unstale":"2024-04-25T13:02:45.897170+0000","last_undegraded":"2024-04-25T13:02:45.897170+0000","last_fullsized":"2024-04-25T13:02:45.897170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:12.960787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.948418+0000","last_change":"2024-04-25T13:02:43.909925+0000","last_active":"2024-04-25T13:02:48.948418+0000","last_peered":"2024-04-25T13:02:48.948418+0000","last_clean":"2024-04-25T13:02:48.948418+0000","last_became_active":"2024-04-25T13:02:43.909602+0000","last_became_peered":"2024-04-25T13:02:43.909602+0000","last_unstale":"2024-04-25T13:02:48.948418+0000","last_undegraded":"2024-04-25T13:02:48.948418+0000","last_fullsized":"2024-04-25T13:02:48.948418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:09.196604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410049+0000","last_change":"2024-04-25T13:02:44.742431+0000","last_active":"2024-04-25T13:02:46.410049+0000","last_peered":"2024-04-25T13:02:46.410049+0000","last_clean":"2024-04-25T13:02:46.410049+0000","last_became_active":"2024-04-25T13:02:44.742120+0000","last_became_peered":"2024-04-25T13:02:44.742120+0000","last_unstale":"2024-04-25T13:02:46.410049+0000","last_undegraded":"2024-04-25T13:02:46.410049+0000","last_fullsized":"2024-04-25T13:02:46.410049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:08.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732789+0000","last_change":"2024-04-25T13:02:43.911342+0000","last_active":"2024-04-25T13:02:45.732789+0000","last_peered":"2024-04-25T13:02:45.732789+0000","last_clean":"2024-04-25T13:02:45.732789+0000","last_became_active":"2024-04-25T13:02:43.910741+0000","last_became_peered":"2024-04-25T13:02:43.910741+0000","last_unstale":"2024-04-25T13:02:45.732789+0000","last_undegraded":"2024-04-25T13:02:45.732789+0000","last_fullsized":"2024-04-25T13:02:45.732789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:28.464712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410923+0000","last_change":"2024-04-25T13:02:44.736639+0000","last_active":"2024-04-25T13:02:46.410923+0000","last_peered":"2024-04-25T13:02:46.410923+0000","last_clean":"2024-04-25T13:02:46.410923+0000","last_became_active":"2024-04-25T13:02:44.736285+0000","last_became_peered":"2024-04-25T13:02:44.736285+0000","last_unstale":"2024-04-25T13:02:46.410923+0000","last_undegraded":"2024-04-25T13:02:46.410923+0000","last_fullsized":"2024-04-25T13:02:46.410923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.840298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410199+0000","last_change":"2024-04-25T13:02:43.906893+0000","last_active":"2024-04-25T13:02:46.410199+0000","last_peered":"2024-04-25T13:02:46.410199+0000","last_clean":"2024-04-25T13:02:46.410199+0000","last_became_active":"2024-04-25T13:02:43.906538+0000","last_became_peered":"2024-04-25T13:02:43.906538+0000","last_unstale":"2024-04-25T13:02:46.410199+0000","last_undegraded":"2024-04-25T13:02:46.410199+0000","last_fullsized":"2024-04-25T13:02:46.410199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:10.601792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410359+0000","last_change":"2024-04-25T13:02:44.741092+0000","last_active":"2024-04-25T13:02:46.410359+0000","last_peered":"2024-04-25T13:02:46.410359+0000","last_clean":"2024-04-25T13:02:46.410359+0000","last_became_active":"2024-04-25T13:02:44.740755+0000","last_became_peered":"2024-04-25T13:02:44.740755+0000","last_unstale":"2024-04-25T13:02:46.410359+0000","last_undegraded":"2024-04-25T13:02:46.410359+0000","last_fullsized":"2024-04-25T13:02:46.410359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:49.124211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.692989+0000","last_change":"2024-04-25T13:02:43.909572+0000","last_active":"2024-04-25T13:02:50.692989+0000","last_peered":"2024-04-25T13:02:50.692989+0000","last_clean":"2024-04-25T13:02:50.692989+0000","last_became_active":"2024-04-25T13:02:43.909109+0000","last_became_peered":"2024-04-25T13:02:43.909109+0000","last_unstale":"2024-04-25T13:02:50.692989+0000","last_undegraded":"2024-04-25T13:02:50.692989+0000","last_fullsized":"2024-04-25T13:02:50.692989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.505599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732701+0000","last_change":"2024-04-25T13:02:44.740972+0000","last_active":"2024-04-25T13:02:45.732701+0000","last_peered":"2024-04-25T13:02:45.732701+0000","last_clean":"2024-04-25T13:02:45.732701+0000","last_became_active":"2024-04-25T13:02:44.740620+0000","last_became_peered":"2024-04-25T13:02:44.740620+0000","last_unstale":"2024-04-25T13:02:45.732701+0000","last_undegraded":"2024-04-25T13:02:45.732701+0000","last_fullsized":"2024-04-25T13:02:45.732701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:45.787763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.694052+0000","last_change":"2024-04-25T13:02:43.901205+0000","last_active":"2024-04-25T13:02:50.694052+0000","last_peered":"2024-04-25T13:02:50.694052+0000","last_clean":"2024-04-25T13:02:50.694052+0000","last_became_active":"2024-04-25T13:02:43.900796+0000","last_became_peered":"2024-04-25T13:02:43.900796+0000","last_unstale":"2024-04-25T13:02:50.694052+0000","last_undegraded":"2024-04-25T13:02:50.694052+0000","last_fullsized":"2024-04-25T13:02:50.694052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:14.020387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410546+0000","last_change":"2024-04-25T13:02:44.739119+0000","last_active":"2024-04-25T13:02:46.410546+0000","last_peered":"2024-04-25T13:02:46.410546+0000","last_clean":"2024-04-25T13:02:46.410546+0000","last_became_active":"2024-04-25T13:02:44.738804+0000","last_became_peered":"2024-04-25T13:02:44.738804+0000","last_unstale":"2024-04-25T13:02:46.410546+0000","last_undegraded":"2024-04-25T13:02:46.410546+0000","last_fullsized":"2024-04-25T13:02:46.410546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:28.424093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897505+0000","last_change":"2024-04-25T13:02:43.911812+0000","last_active":"2024-04-25T13:02:45.897505+0000","last_peered":"2024-04-25T13:02:45.897505+0000","last_clean":"2024-04-25T13:02:45.897505+0000","last_became_active":"2024-04-25T13:02:43.911404+0000","last_became_peered":"2024-04-25T13:02:43.911404+0000","last_unstale":"2024-04-25T13:02:45.897505+0000","last_undegraded":"2024-04-25T13:02:45.897505+0000","last_fullsized":"2024-04-25T13:02:45.897505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:17.210980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036573+0000","last_change":"2024-04-25T13:02:44.732546+0000","last_active":"2024-04-25T13:02:46.036573+0000","last_peered":"2024-04-25T13:02:46.036573+0000","last_clean":"2024-04-25T13:02:46.036573+0000","last_became_active":"2024-04-25T13:02:44.732238+0000","last_became_peered":"2024-04-25T13:02:44.732238+0000","last_unstale":"2024-04-25T13:02:46.036573+0000","last_undegraded":"2024-04-25T13:02:46.036573+0000","last_fullsized":"2024-04-25T13:02:46.036573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.293939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410692+0000","last_change":"2024-04-25T13:02:43.910620+0000","last_active":"2024-04-25T13:02:46.410692+0000","last_peered":"2024-04-25T13:02:46.410692+0000","last_clean":"2024-04-25T13:02:46.410692+0000","last_became_active":"2024-04-25T13:02:43.910325+0000","last_became_peered":"2024-04-25T13:02:43.910325+0000","last_unstale":"2024-04-25T13:02:46.410692+0000","last_undegraded":"2024-04-25T13:02:46.410692+0000","last_fullsized":"2024-04-25T13:02:46.410692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.185209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036326+0000","last_change":"2024-04-25T13:02:44.737134+0000","last_active":"2024-04-25T13:02:46.036326+0000","last_peered":"2024-04-25T13:02:46.036326+0000","last_clean":"2024-04-25T13:02:46.036326+0000","last_became_active":"2024-04-25T13:02:44.736790+0000","last_became_peered":"2024-04-25T13:02:44.736790+0000","last_unstale":"2024-04-25T13:02:46.036326+0000","last_undegraded":"2024-04-25T13:02:46.036326+0000","last_fullsized":"2024-04-25T13:02:46.036326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:02.680752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732933+0000","last_change":"2024-04-25T13:02:44.740743+0000","last_active":"2024-04-25T13:02:45.732933+0000","last_peered":"2024-04-25T13:02:45.732933+0000","last_clean":"2024-04-25T13:02:45.732933+0000","last_became_active":"2024-04-25T13:02:44.740421+0000","last_became_peered":"2024-04-25T13:02:44.740421+0000","last_unstale":"2024-04-25T13:02:45.732933+0000","last_undegraded":"2024-04-25T13:02:45.732933+0000","last_fullsized":"2024-04-25T13:02:45.732933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950953+0000","last_change":"2024-04-25T13:02:43.920678+0000","last_active":"2024-04-25T13:02:48.950953+0000","last_peered":"2024-04-25T13:02:48.950953+0000","last_clean":"2024-04-25T13:02:48.950953+0000","last_became_active":"2024-04-25T13:02:43.920338+0000","last_became_peered":"2024-04-25T13:02:43.920338+0000","last_unstale":"2024-04-25T13:02:48.950953+0000","last_undegraded":"2024-04-25T13:02:48.950953+0000","last_fullsized":"2024-04-25T13:02:48.950953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732583+0000","last_change":"2024-04-25T13:02:44.736655+0000","last_active":"2024-04-25T13:02:45.732583+0000","last_peered":"2024-04-25T13:02:45.732583+0000","last_clean":"2024-04-25T13:02:45.732583+0000","last_became_active":"2024-04-25T13:02:44.736340+0000","last_became_peered":"2024-04-25T13:02:44.736340+0000","last_unstale":"2024-04-25T13:02:45.732583+0000","last_undegraded":"2024-04-25T13:02:45.732583+0000","last_fullsized":"2024-04-25T13:02:45.732583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:35.129606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410403+0000","last_change":"2024-04-25T13:02:43.903179+0000","last_active":"2024-04-25T13:02:46.410403+0000","last_peered":"2024-04-25T13:02:46.410403+0000","last_clean":"2024-04-25T13:02:46.410403+0000","last_became_active":"2024-04-25T13:02:43.902799+0000","last_became_peered":"2024-04-25T13:02:43.902799+0000","last_unstale":"2024-04-25T13:02:46.410403+0000","last_undegraded":"2024-04-25T13:02:46.410403+0000","last_fullsized":"2024-04-25T13:02:46.410403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.472772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410553+0000","last_change":"2024-04-25T13:01:33.625244+0000","last_active":"2024-04-25T13:02:46.410553+0000","last_peered":"2024-04-25T13:02:46.410553+0000","last_clean":"2024-04-25T13:02:46.410553+0000","last_became_active":"2024-04-25T13:01:33.624760+0000","last_became_peered":"2024-04-25T13:01:33.624760+0000","last_unstale":"2024-04-25T13:02:46.410553+0000","last_undegraded":"2024-04-25T13:02:46.410553+0000","last_fullsized":"2024-04-25T13:02:46.410553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_clean_scrub_stamp":"2024-04-25T12:58:01.939677+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:21.636432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.409923+0000","last_change":"2024-04-25T13:02:44.732711+0000","last_active":"2024-04-25T13:02:46.409923+0000","last_peered":"2024-04-25T13:02:46.409923+0000","last_clean":"2024-04-25T13:02:46.409923+0000","last_became_active":"2024-04-25T13:02:44.732394+0000","last_became_peered":"2024-04-25T13:02:44.732394+0000","last_unstale":"2024-04-25T13:02:46.409923+0000","last_undegraded":"2024-04-25T13:02:46.409923+0000","last_fullsized":"2024-04-25T13:02:46.409923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:02.601328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950449+0000","last_change":"2024-04-25T13:02:43.908773+0000","last_active":"2024-04-25T13:02:48.950449+0000","last_peered":"2024-04-25T13:02:48.950449+0000","last_clean":"2024-04-25T13:02:48.950449+0000","last_became_active":"2024-04-25T13:02:43.908440+0000","last_became_peered":"2024-04-25T13:02:43.908440+0000","last_unstale":"2024-04-25T13:02:48.950449+0000","last_undegraded":"2024-04-25T13:02:48.950449+0000","last_fullsized":"2024-04-25T13:02:48.950449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:49.565391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.693524+0000","last_change":"2024-04-25T13:02:43.908686+0000","last_active":"2024-04-25T13:02:50.693524+0000","last_peered":"2024-04-25T13:02:50.693524+0000","last_clean":"2024-04-25T13:02:50.693524+0000","last_became_active":"2024-04-25T13:02:43.908391+0000","last_became_peered":"2024-04-25T13:02:43.908391+0000","last_unstale":"2024-04-25T13:02:50.693524+0000","last_undegraded":"2024-04-25T13:02:50.693524+0000","last_fullsized":"2024-04-25T13:02:50.693524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:49.938946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410244+0000","last_change":"2024-04-25T13:02:44.730629+0000","last_active":"2024-04-25T13:02:46.410244+0000","last_peered":"2024-04-25T13:02:46.410244+0000","last_clean":"2024-04-25T13:02:46.410244+0000","last_became_active":"2024-04-25T13:02:44.730288+0000","last_became_peered":"2024-04-25T13:02:44.730288+0000","last_unstale":"2024-04-25T13:02:46.410244+0000","last_undegraded":"2024-04-25T13:02:46.410244+0000","last_fullsized":"2024-04-25T13:02:46.410244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:26.255183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950741+0000","last_change":"2024-04-25T13:02:43.918791+0000","last_active":"2024-04-25T13:02:48.950741+0000","last_peered":"2024-04-25T13:02:48.950741+0000","last_clean":"2024-04-25T13:02:48.950741+0000","last_became_active":"2024-04-25T13:02:43.918454+0000","last_became_peered":"2024-04-25T13:02:43.918454+0000","last_unstale":"2024-04-25T13:02:48.950741+0000","last_undegraded":"2024-04-25T13:02:48.950741+0000","last_fullsized":"2024-04-25T13:02:48.950741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.137235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733010+0000","last_change":"2024-04-25T13:02:44.740324+0000","last_active":"2024-04-25T13:02:45.733010+0000","last_peered":"2024-04-25T13:02:45.733010+0000","last_clean":"2024-04-25T13:02:45.733010+0000","last_became_active":"2024-04-25T13:02:44.739993+0000","last_became_peered":"2024-04-25T13:02:44.739993+0000","last_unstale":"2024-04-25T13:02:45.733010+0000","last_undegraded":"2024-04-25T13:02:45.733010+0000","last_fullsized":"2024-04-25T13:02:45.733010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.207392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410228+0000","last_change":"2024-04-25T13:02:43.903356+0000","last_active":"2024-04-25T13:02:46.410228+0000","last_peered":"2024-04-25T13:02:46.410228+0000","last_clean":"2024-04-25T13:02:46.410228+0000","last_became_active":"2024-04-25T13:02:43.903046+0000","last_became_peered":"2024-04-25T13:02:43.903046+0000","last_unstale":"2024-04-25T13:02:46.410228+0000","last_undegraded":"2024-04-25T13:02:46.410228+0000","last_fullsized":"2024-04-25T13:02:46.410228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.514373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284851+0000","last_change":"2024-04-25T13:02:44.745033+0000","last_active":"2024-04-25T13:02:46.284851+0000","last_peered":"2024-04-25T13:02:46.284851+0000","last_clean":"2024-04-25T13:02:46.284851+0000","last_became_active":"2024-04-25T13:02:44.744718+0000","last_became_peered":"2024-04-25T13:02:44.744718+0000","last_unstale":"2024-04-25T13:02:46.284851+0000","last_undegraded":"2024-04-25T13:02:46.284851+0000","last_fullsized":"2024-04-25T13:02:46.284851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:26.736807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732494+0000","last_change":"2024-04-25T13:02:43.909366+0000","last_active":"2024-04-25T13:02:45.732494+0000","last_peered":"2024-04-25T13:02:45.732494+0000","last_clean":"2024-04-25T13:02:45.732494+0000","last_became_active":"2024-04-25T13:02:43.908968+0000","last_became_peered":"2024-04-25T13:02:43.908968+0000","last_unstale":"2024-04-25T13:02:45.732494+0000","last_undegraded":"2024-04-25T13:02:45.732494+0000","last_fullsized":"2024-04-25T13:02:45.732494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:36.915466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.896963+0000","last_change":"2024-04-25T13:02:44.730481+0000","last_active":"2024-04-25T13:02:45.896963+0000","last_peered":"2024-04-25T13:02:45.896963+0000","last_clean":"2024-04-25T13:02:45.896963+0000","last_became_active":"2024-04-25T13:02:44.730163+0000","last_became_peered":"2024-04-25T13:02:44.730163+0000","last_unstale":"2024-04-25T13:02:45.896963+0000","last_undegraded":"2024-04-25T13:02:45.896963+0000","last_fullsized":"2024-04-25T13:02:45.896963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:12.207840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:49.968452+0000","last_change":"2024-04-25T13:02:43.922262+0000","last_active":"2024-04-25T13:02:49.968452+0000","last_peered":"2024-04-25T13:02:49.968452+0000","last_clean":"2024-04-25T13:02:49.968452+0000","last_became_active":"2024-04-25T13:02:43.921356+0000","last_became_peered":"2024-04-25T13:02:43.921356+0000","last_unstale":"2024-04-25T13:02:49.968452+0000","last_undegraded":"2024-04-25T13:02:49.968452+0000","last_fullsized":"2024-04-25T13:02:49.968452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:27.446112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410083+0000","last_change":"2024-04-25T13:02:44.735879+0000","last_active":"2024-04-25T13:02:46.410083+0000","last_peered":"2024-04-25T13:02:46.410083+0000","last_clean":"2024-04-25T13:02:46.410083+0000","last_became_active":"2024-04-25T13:02:44.735574+0000","last_became_peered":"2024-04-25T13:02:44.735574+0000","last_unstale":"2024-04-25T13:02:46.410083+0000","last_undegraded":"2024-04-25T13:02:46.410083+0000","last_fullsized":"2024-04-25T13:02:46.410083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:14.094813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410077+0000","last_change":"2024-04-25T13:02:43.912250+0000","last_active":"2024-04-25T13:02:46.410077+0000","last_peered":"2024-04-25T13:02:46.410077+0000","last_clean":"2024-04-25T13:02:46.410077+0000","last_became_active":"2024-04-25T13:02:43.911945+0000","last_became_peered":"2024-04-25T13:02:43.911945+0000","last_unstale":"2024-04-25T13:02:46.410077+0000","last_undegraded":"2024-04-25T13:02:46.410077+0000","last_fullsized":"2024-04-25T13:02:46.410077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:51.872195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732542+0000","last_change":"2024-04-25T13:02:44.739184+0000","last_active":"2024-04-25T13:02:45.732542+0000","last_peered":"2024-04-25T13:02:45.732542+0000","last_clean":"2024-04-25T13:02:45.732542+0000","last_became_active":"2024-04-25T13:02:44.738862+0000","last_became_peered":"2024-04-25T13:02:44.738862+0000","last_unstale":"2024-04-25T13:02:45.732542+0000","last_undegraded":"2024-04-25T13:02:45.732542+0000","last_fullsized":"2024-04-25T13:02:45.732542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.605730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.940196+0000","last_change":"2024-04-25T13:02:43.910735+0000","last_active":"2024-04-25T13:02:47.940196+0000","last_peered":"2024-04-25T13:02:47.940196+0000","last_clean":"2024-04-25T13:02:47.940196+0000","last_became_active":"2024-04-25T13:02:43.910110+0000","last_became_peered":"2024-04-25T13:02:43.910110+0000","last_unstale":"2024-04-25T13:02:47.940196+0000","last_undegraded":"2024-04-25T13:02:47.940196+0000","last_fullsized":"2024-04-25T13:02:47.940196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:49.114560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410957+0000","last_change":"2024-04-25T13:02:44.738410+0000","last_active":"2024-04-25T13:02:46.410957+0000","last_peered":"2024-04-25T13:02:46.410957+0000","last_clean":"2024-04-25T13:02:46.410957+0000","last_became_active":"2024-04-25T13:02:44.738091+0000","last_became_peered":"2024-04-25T13:02:44.738091+0000","last_unstale":"2024-04-25T13:02:46.410957+0000","last_undegraded":"2024-04-25T13:02:46.410957+0000","last_fullsized":"2024-04-25T13:02:46.410957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.737907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.952272+0000","last_change":"2024-04-25T13:02:43.914563+0000","last_active":"2024-04-25T13:02:48.952272+0000","last_peered":"2024-04-25T13:02:48.952272+0000","last_clean":"2024-04-25T13:02:48.952272+0000","last_became_active":"2024-04-25T13:02:43.914274+0000","last_became_peered":"2024-04-25T13:02:43.914274+0000","last_unstale":"2024-04-25T13:02:48.952272+0000","last_undegraded":"2024-04-25T13:02:48.952272+0000","last_fullsized":"2024-04-25T13:02:48.952272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.411040+0000","last_change":"2024-04-25T13:02:44.730098+0000","last_active":"2024-04-25T13:02:46.411040+0000","last_peered":"2024-04-25T13:02:46.411040+0000","last_clean":"2024-04-25T13:02:46.411040+0000","last_became_active":"2024-04-25T13:02:44.729656+0000","last_became_peered":"2024-04-25T13:02:44.729656+0000","last_unstale":"2024-04-25T13:02:46.411040+0000","last_undegraded":"2024-04-25T13:02:46.411040+0000","last_fullsized":"2024-04-25T13:02:46.411040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.344967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732984+0000","last_change":"2024-04-25T13:02:43.911593+0000","last_active":"2024-04-25T13:02:45.732984+0000","last_peered":"2024-04-25T13:02:45.732984+0000","last_clean":"2024-04-25T13:02:45.732984+0000","last_became_active":"2024-04-25T13:02:43.910484+0000","last_became_peered":"2024-04-25T13:02:43.910484+0000","last_unstale":"2024-04-25T13:02:45.732984+0000","last_undegraded":"2024-04-25T13:02:45.732984+0000","last_fullsized":"2024-04-25T13:02:45.732984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:52.257269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410898+0000","last_change":"2024-04-25T13:02:44.737141+0000","last_active":"2024-04-25T13:02:46.410898+0000","last_peered":"2024-04-25T13:02:46.410898+0000","last_clean":"2024-04-25T13:02:46.410898+0000","last_became_active":"2024-04-25T13:02:44.736464+0000","last_became_peered":"2024-04-25T13:02:44.736464+0000","last_unstale":"2024-04-25T13:02:46.410898+0000","last_undegraded":"2024-04-25T13:02:46.410898+0000","last_fullsized":"2024-04-25T13:02:46.410898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:16.337344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410217+0000","last_change":"2024-04-25T13:02:43.921129+0000","last_active":"2024-04-25T13:02:46.410217+0000","last_peered":"2024-04-25T13:02:46.410217+0000","last_clean":"2024-04-25T13:02:46.410217+0000","last_became_active":"2024-04-25T13:02:43.920768+0000","last_became_peered":"2024-04-25T13:02:43.920768+0000","last_unstale":"2024-04-25T13:02:46.410217+0000","last_undegraded":"2024-04-25T13:02:46.410217+0000","last_fullsized":"2024-04-25T13:02:46.410217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:49.178313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.411014+0000","last_change":"2024-04-25T13:02:44.731934+0000","last_active":"2024-04-25T13:02:46.411014+0000","last_peered":"2024-04-25T13:02:46.411014+0000","last_clean":"2024-04-25T13:02:46.411014+0000","last_became_active":"2024-04-25T13:02:44.731624+0000","last_became_peered":"2024-04-25T13:02:44.731624+0000","last_unstale":"2024-04-25T13:02:46.411014+0000","last_undegraded":"2024-04-25T13:02:46.411014+0000","last_fullsized":"2024-04-25T13:02:46.411014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.809419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410087+0000","last_change":"2024-04-25T13:02:44.736982+0000","last_active":"2024-04-25T13:02:46.410087+0000","last_peered":"2024-04-25T13:02:46.410087+0000","last_clean":"2024-04-25T13:02:46.410087+0000","last_became_active":"2024-04-25T13:02:44.735127+0000","last_became_peered":"2024-04-25T13:02:44.735127+0000","last_unstale":"2024-04-25T13:02:46.410087+0000","last_undegraded":"2024-04-25T13:02:46.410087+0000","last_fullsized":"2024-04-25T13:02:46.410087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:06.827686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.409910+0000","last_change":"2024-04-25T13:02:44.737994+0000","last_active":"2024-04-25T13:02:46.409910+0000","last_peered":"2024-04-25T13:02:46.409910+0000","last_clean":"2024-04-25T13:02:46.409910+0000","last_became_active":"2024-04-25T13:02:44.737439+0000","last_became_peered":"2024-04-25T13:02:44.737439+0000","last_unstale":"2024-04-25T13:02:46.409910+0000","last_undegraded":"2024-04-25T13:02:46.409910+0000","last_fullsized":"2024-04-25T13:02:46.409910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:53.755757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410151+0000","last_change":"2024-04-25T13:02:44.739660+0000","last_active":"2024-04-25T13:02:46.410151+0000","last_peered":"2024-04-25T13:02:46.410151+0000","last_clean":"2024-04-25T13:02:46.410151+0000","last_became_active":"2024-04-25T13:02:44.738553+0000","last_became_peered":"2024-04-25T13:02:44.738553+0000","last_unstale":"2024-04-25T13:02:46.410151+0000","last_undegraded":"2024-04-25T13:02:46.410151+0000","last_fullsized":"2024-04-25T13:02:46.410151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:07.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284578+0000","last_change":"2024-04-25T13:02:44.736644+0000","last_active":"2024-04-25T13:02:46.284578+0000","last_peered":"2024-04-25T13:02:46.284578+0000","last_clean":"2024-04-25T13:02:46.284578+0000","last_became_active":"2024-04-25T13:02:44.736325+0000","last_became_peered":"2024-04-25T13:02:44.736325+0000","last_unstale":"2024-04-25T13:02:46.284578+0000","last_undegraded":"2024-04-25T13:02:46.284578+0000","last_fullsized":"2024-04-25T13:02:46.284578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:34.102364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410279+0000","last_change":"2024-04-25T13:02:44.739324+0000","last_active":"2024-04-25T13:02:46.410279+0000","last_peered":"2024-04-25T13:02:46.410279+0000","last_clean":"2024-04-25T13:02:46.410279+0000","last_became_active":"2024-04-25T13:02:44.738994+0000","last_became_peered":"2024-04-25T13:02:44.738994+0000","last_unstale":"2024-04-25T13:02:46.410279+0000","last_undegraded":"2024-04-25T13:02:46.410279+0000","last_fullsized":"2024-04-25T13:02:46.410279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:50.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410481+0000","last_change":"2024-04-25T13:02:44.738075+0000","last_active":"2024-04-25T13:02:46.410481+0000","last_peered":"2024-04-25T13:02:46.410481+0000","last_clean":"2024-04-25T13:02:46.410481+0000","last_became_active":"2024-04-25T13:02:44.736616+0000","last_became_peered":"2024-04-25T13:02:44.736616+0000","last_unstale":"2024-04-25T13:02:46.410481+0000","last_undegraded":"2024-04-25T13:02:46.410481+0000","last_fullsized":"2024-04-25T13:02:46.410481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:14.875245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410401+0000","last_change":"2024-04-25T13:02:44.730926+0000","last_active":"2024-04-25T13:02:46.410401+0000","last_peered":"2024-04-25T13:02:46.410401+0000","last_clean":"2024-04-25T13:02:46.410401+0000","last_became_active":"2024-04-25T13:02:44.730154+0000","last_became_peered":"2024-04-25T13:02:44.730154+0000","last_unstale":"2024-04-25T13:02:46.410401+0000","last_undegraded":"2024-04-25T13:02:46.410401+0000","last_fullsized":"2024-04-25T13:02:46.410401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:07.361869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732990+0000","last_change":"2024-04-25T13:02:44.740104+0000","last_active":"2024-04-25T13:02:45.732990+0000","last_peered":"2024-04-25T13:02:45.732990+0000","last_clean":"2024-04-25T13:02:45.732990+0000","last_became_active":"2024-04-25T13:02:44.739570+0000","last_became_peered":"2024-04-25T13:02:44.739570+0000","last_unstale":"2024-04-25T13:02:45.732990+0000","last_undegraded":"2024-04-25T13:02:45.732990+0000","last_fullsized":"2024-04-25T13:02:45.732990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:25.862867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732730+0000","last_change":"2024-04-25T13:02:44.735555+0000","last_active":"2024-04-25T13:02:45.732730+0000","last_peered":"2024-04-25T13:02:45.732730+0000","last_clean":"2024-04-25T13:02:45.732730+0000","last_became_active":"2024-04-25T13:02:44.735238+0000","last_became_peered":"2024-04-25T13:02:44.735238+0000","last_unstale":"2024-04-25T13:02:45.732730+0000","last_undegraded":"2024-04-25T13:02:45.732730+0000","last_fullsized":"2024-04-25T13:02:45.732730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.611943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732283+0000","last_change":"2024-04-25T13:02:44.730959+0000","last_active":"2024-04-25T13:02:45.732283+0000","last_peered":"2024-04-25T13:02:45.732283+0000","last_clean":"2024-04-25T13:02:45.732283+0000","last_became_active":"2024-04-25T13:02:44.730583+0000","last_became_peered":"2024-04-25T13:02:44.730583+0000","last_unstale":"2024-04-25T13:02:45.732283+0000","last_undegraded":"2024-04-25T13:02:45.732283+0000","last_fullsized":"2024-04-25T13:02:45.732283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.527837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732728+0000","last_change":"2024-04-25T13:02:44.734174+0000","last_active":"2024-04-25T13:02:45.732728+0000","last_peered":"2024-04-25T13:02:45.732728+0000","last_clean":"2024-04-25T13:02:45.732728+0000","last_became_active":"2024-04-25T13:02:44.733785+0000","last_became_peered":"2024-04-25T13:02:44.733785+0000","last_unstale":"2024-04-25T13:02:45.732728+0000","last_undegraded":"2024-04-25T13:02:45.732728+0000","last_fullsized":"2024-04-25T13:02:45.732728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:39.641206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410278+0000","last_change":"2024-04-25T13:02:44.740058+0000","last_active":"2024-04-25T13:02:46.410278+0000","last_peered":"2024-04-25T13:02:46.410278+0000","last_clean":"2024-04-25T13:02:46.410278+0000","last_became_active":"2024-04-25T13:02:44.739716+0000","last_became_peered":"2024-04-25T13:02:44.739716+0000","last_unstale":"2024-04-25T13:02:46.410278+0000","last_undegraded":"2024-04-25T13:02:46.410278+0000","last_fullsized":"2024-04-25T13:02:46.410278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:21.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410318+0000","last_change":"2024-04-25T13:02:44.731335+0000","last_active":"2024-04-25T13:02:46.410318+0000","last_peered":"2024-04-25T13:02:46.410318+0000","last_clean":"2024-04-25T13:02:46.410318+0000","last_became_active":"2024-04-25T13:02:44.731017+0000","last_became_peered":"2024-04-25T13:02:44.731017+0000","last_unstale":"2024-04-25T13:02:46.410318+0000","last_undegraded":"2024-04-25T13:02:46.410318+0000","last_fullsized":"2024-04-25T13:02:46.410318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:29.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":288217,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4068,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":128,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":13338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1034404,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":444534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":441980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":1032122,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:02:52.792 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:02:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 -- 172.21.15.38:0/3056159866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681011e0 msgr2=0x7fdd68101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/3056159866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681011e0 0x7fdd68101640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009900 tx=0x7fdd5802a480 comp rx=0 tx=0).stop 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 -- 172.21.15.38:0/3056159866 shutdown_connections 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/3056159866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd68101b80 0x7fdd68102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/3056159866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681011e0 0x7fdd68101640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/3056159866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd68100880 0x7fdd68100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 -- 172.21.15.38:0/3056159866 >> 172.21.15.38:0/3056159866 conn(0x7fdd680fc400 msgr2=0x7fdd680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 -- 172.21.15.38:0/3056159866 shutdown_connections 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.932+0000 7fdd6fca7640 1 -- 172.21.15.38:0/3056159866 wait complete. 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 Processor -- start 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 -- start start 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 0x7fdd6819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd681011e0 0x7fdd6819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd68101b80 0x7fdd681a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6810fde0 con 0x7fdd68100880 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6810fc60 con 0x7fdd681011e0 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6fca7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6810ff60 con 0x7fdd68101b80 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 0x7fdd6819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 0x7fdd6819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37662/0 (socket says 172.21.15.38:37662) 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 -- 172.21.15.38:0/1492131010 learned_addr learned my addr 172.21.15.38:0/1492131010 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6d242640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd681011e0 0x7fdd6819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6e244640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd68101b80 0x7fdd681a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 -- 172.21.15.38:0/1492131010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd68101b80 msgr2=0x7fdd681a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd68101b80 0x7fdd681a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 -- 172.21.15.38:0/1492131010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd681011e0 msgr2=0x7fdd6819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:52.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.933+0000 7fdd6da43640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd681011e0 0x7fdd6819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd6da43640 1 -- 172.21.15.38:0/1492131010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd681a20d0 con 0x7fdd68100880 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd6da43640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 0x7fdd6819a4b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d950 tx=0x7fdd6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64017030 con 0x7fdd68100880 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd681a23c0 con 0x7fdd68100880 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd64004020 con 0x7fdd68100880 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd640047a0 con 0x7fdd68100880 2024-04-25T13:02:52.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.934+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd681a2930 con 0x7fdd68100880 2024-04-25T13:02:52.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.935+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6810b7c0 con 0x7fdd68100880 2024-04-25T13:02:52.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.936+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdd640041d0 con 0x7fdd68100880 2024-04-25T13:02:52.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.936+0000 7fdd5effd640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd540a3c40 0x7fdd540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:52.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.937+0000 7fdd6d242640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd540a3c40 0x7fdd540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:52.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.937+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdd640cfe70 con 0x7fdd68100880 2024-04-25T13:02:52.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.937+0000 7fdd6d242640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd540a3c40 0x7fdd540a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c00 tx=0x7fdd58031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:52.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:52.941+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd6409c8a0 con 0x7fdd68100880 2024-04-25T13:02:53.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:52 smithi038 ceph-mon[29400]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 969 B/s rd, 3.9 KiB/s wr, 18 op/s 2024-04-25T13:02:53.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:52 smithi038 ceph-mon[29400]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:52 smithi038 ceph-mon[29400]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:52 smithi038 ceph-mon[29400]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:02:53.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:52 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/439836554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:53.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:52 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3947790792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:53.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.124+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd68062390 con 0x7fdd540a3c40 2024-04-25T13:02:53.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.132+0000 7fdd5effd640 1 -- 172.21.15.38:0/1492131010 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258879 (secure 0 0 0) 0x7fdd68062390 con 0x7fdd540a3c40 2024-04-25T13:02:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:53.134 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T13:02:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd540a3c40 msgr2=0x7fdd540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd540a3c40 0x7fdd540a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c00 tx=0x7fdd58031510 comp rx=0 tx=0).stop 2024-04-25T13:02:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 msgr2=0x7fdd6819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 0x7fdd6819a4b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d950 tx=0x7fdd6400de10 comp rx=0 tx=0).stop 2024-04-25T13:02:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 shutdown_connections 2024-04-25T13:02:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd540a3c40 0x7fdd540a6100 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd68101b80 0x7fdd681a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd681011e0 0x7fdd6819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 --2- 172.21.15.38:0/1492131010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68100880 0x7fdd6819a4b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 >> 172.21.15.38:0/1492131010 conn(0x7fdd680fc400 msgr2=0x7fdd680fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 shutdown_connections 2024-04-25T13:02:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:53.135+0000 7fdd6fca7640 1 -- 172.21.15.38:0/1492131010 wait complete. 2024-04-25T13:02:53.144 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-25T13:02:51.282414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594436,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":143,"num_read_kb":126,"num_write":309,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372836,"kb_used_data":11148,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568988,"statfs":{"total":1151940427776,"available":1151558643712,"internally_reserved":0,"allocated":11415552,"data_stored":7082544,"data_compressed":109923,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4068,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":128,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.391233"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410619+0000","last_change":"2024-04-25T13:02:44.736489+0000","last_active":"2024-04-25T13:02:46.410619+0000","last_peered":"2024-04-25T13:02:46.410619+0000","last_clean":"2024-04-25T13:02:46.410619+0000","last_became_active":"2024-04-25T13:02:44.735677+0000","last_became_peered":"2024-04-25T13:02:44.735677+0000","last_unstale":"2024-04-25T13:02:46.410619+0000","last_undegraded":"2024-04-25T13:02:46.410619+0000","last_fullsized":"2024-04-25T13:02:46.410619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:59.810878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284870+0000","last_change":"2024-04-25T13:02:44.740349+0000","last_active":"2024-04-25T13:02:46.284870+0000","last_peered":"2024-04-25T13:02:46.284870+0000","last_clean":"2024-04-25T13:02:46.284870+0000","last_became_active":"2024-04-25T13:02:44.740024+0000","last_became_peered":"2024-04-25T13:02:44.740024+0000","last_unstale":"2024-04-25T13:02:46.284870+0000","last_undegraded":"2024-04-25T13:02:46.284870+0000","last_fullsized":"2024-04-25T13:02:46.284870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:35.763278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284734+0000","last_change":"2024-04-25T13:02:44.742136+0000","last_active":"2024-04-25T13:02:46.284734+0000","last_peered":"2024-04-25T13:02:46.284734+0000","last_clean":"2024-04-25T13:02:46.284734+0000","last_became_active":"2024-04-25T13:02:44.741840+0000","last_became_peered":"2024-04-25T13:02:44.741840+0000","last_unstale":"2024-04-25T13:02:46.284734+0000","last_undegraded":"2024-04-25T13:02:46.284734+0000","last_fullsized":"2024-04-25T13:02:46.284734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:48.690435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410346+0000","last_change":"2024-04-25T13:02:44.733349+0000","last_active":"2024-04-25T13:02:46.410346+0000","last_peered":"2024-04-25T13:02:46.410346+0000","last_clean":"2024-04-25T13:02:46.410346+0000","last_became_active":"2024-04-25T13:02:44.732389+0000","last_became_peered":"2024-04-25T13:02:44.732389+0000","last_unstale":"2024-04-25T13:02:46.410346+0000","last_undegraded":"2024-04-25T13:02:46.410346+0000","last_fullsized":"2024-04-25T13:02:46.410346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:28.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.896983+0000","last_change":"2024-04-25T13:02:44.732852+0000","last_active":"2024-04-25T13:02:45.896983+0000","last_peered":"2024-04-25T13:02:45.896983+0000","last_clean":"2024-04-25T13:02:45.896983+0000","last_became_active":"2024-04-25T13:02:44.732536+0000","last_became_peered":"2024-04-25T13:02:44.732536+0000","last_unstale":"2024-04-25T13:02:45.896983+0000","last_undegraded":"2024-04-25T13:02:45.896983+0000","last_fullsized":"2024-04-25T13:02:45.896983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:58.358684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036405+0000","last_change":"2024-04-25T13:02:44.731636+0000","last_active":"2024-04-25T13:02:46.036405+0000","last_peered":"2024-04-25T13:02:46.036405+0000","last_clean":"2024-04-25T13:02:46.036405+0000","last_became_active":"2024-04-25T13:02:44.731334+0000","last_became_peered":"2024-04-25T13:02:44.731334+0000","last_unstale":"2024-04-25T13:02:46.036405+0000","last_undegraded":"2024-04-25T13:02:46.036405+0000","last_fullsized":"2024-04-25T13:02:46.036405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:00.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410389+0000","last_change":"2024-04-25T13:02:44.737275+0000","last_active":"2024-04-25T13:02:46.410389+0000","last_peered":"2024-04-25T13:02:46.410389+0000","last_clean":"2024-04-25T13:02:46.410389+0000","last_became_active":"2024-04-25T13:02:44.736924+0000","last_became_peered":"2024-04-25T13:02:44.736924+0000","last_unstale":"2024-04-25T13:02:46.410389+0000","last_undegraded":"2024-04-25T13:02:46.410389+0000","last_fullsized":"2024-04-25T13:02:46.410389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.564465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732223+0000","last_change":"2024-04-25T13:02:44.736736+0000","last_active":"2024-04-25T13:02:45.732223+0000","last_peered":"2024-04-25T13:02:45.732223+0000","last_clean":"2024-04-25T13:02:45.732223+0000","last_became_active":"2024-04-25T13:02:44.736403+0000","last_became_peered":"2024-04-25T13:02:44.736403+0000","last_unstale":"2024-04-25T13:02:45.732223+0000","last_undegraded":"2024-04-25T13:02:45.732223+0000","last_fullsized":"2024-04-25T13:02:45.732223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:54.531475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733075+0000","last_change":"2024-04-25T13:02:44.738124+0000","last_active":"2024-04-25T13:02:45.733075+0000","last_peered":"2024-04-25T13:02:45.733075+0000","last_clean":"2024-04-25T13:02:45.733075+0000","last_became_active":"2024-04-25T13:02:44.737572+0000","last_became_peered":"2024-04-25T13:02:44.737572+0000","last_unstale":"2024-04-25T13:02:45.733075+0000","last_undegraded":"2024-04-25T13:02:45.733075+0000","last_fullsized":"2024-04-25T13:02:45.733075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:00.692786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036657+0000","last_change":"2024-04-25T13:02:44.730108+0000","last_active":"2024-04-25T13:02:46.036657+0000","last_peered":"2024-04-25T13:02:46.036657+0000","last_clean":"2024-04-25T13:02:46.036657+0000","last_became_active":"2024-04-25T13:02:44.729449+0000","last_became_peered":"2024-04-25T13:02:44.729449+0000","last_unstale":"2024-04-25T13:02:46.036657+0000","last_undegraded":"2024-04-25T13:02:46.036657+0000","last_fullsized":"2024-04-25T13:02:46.036657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:19.888190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.411032+0000","last_change":"2024-04-25T13:02:44.730902+0000","last_active":"2024-04-25T13:02:46.411032+0000","last_peered":"2024-04-25T13:02:46.411032+0000","last_clean":"2024-04-25T13:02:46.411032+0000","last_became_active":"2024-04-25T13:02:44.730560+0000","last_became_peered":"2024-04-25T13:02:44.730560+0000","last_unstale":"2024-04-25T13:02:46.411032+0000","last_undegraded":"2024-04-25T13:02:46.411032+0000","last_fullsized":"2024-04-25T13:02:46.411032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:07.133221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410371+0000","last_change":"2024-04-25T13:02:44.730966+0000","last_active":"2024-04-25T13:02:46.410371+0000","last_peered":"2024-04-25T13:02:46.410371+0000","last_clean":"2024-04-25T13:02:46.410371+0000","last_became_active":"2024-04-25T13:02:44.730546+0000","last_became_peered":"2024-04-25T13:02:44.730546+0000","last_unstale":"2024-04-25T13:02:46.410371+0000","last_undegraded":"2024-04-25T13:02:46.410371+0000","last_fullsized":"2024-04-25T13:02:46.410371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.188491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410202+0000","last_change":"2024-04-25T13:02:44.739301+0000","last_active":"2024-04-25T13:02:46.410202+0000","last_peered":"2024-04-25T13:02:46.410202+0000","last_clean":"2024-04-25T13:02:46.410202+0000","last_became_active":"2024-04-25T13:02:44.738704+0000","last_became_peered":"2024-04-25T13:02:44.738704+0000","last_unstale":"2024-04-25T13:02:46.410202+0000","last_undegraded":"2024-04-25T13:02:46.410202+0000","last_fullsized":"2024-04-25T13:02:46.410202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:44.373522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284491+0000","last_change":"2024-04-25T13:02:44.738703+0000","last_active":"2024-04-25T13:02:46.284491+0000","last_peered":"2024-04-25T13:02:46.284491+0000","last_clean":"2024-04-25T13:02:46.284491+0000","last_became_active":"2024-04-25T13:02:44.738373+0000","last_became_peered":"2024-04-25T13:02:44.738373+0000","last_unstale":"2024-04-25T13:02:46.284491+0000","last_undegraded":"2024-04-25T13:02:46.284491+0000","last_fullsized":"2024-04-25T13:02:46.284491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:29.106615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284536+0000","last_change":"2024-04-25T13:02:44.743336+0000","last_active":"2024-04-25T13:02:46.284536+0000","last_peered":"2024-04-25T13:02:46.284536+0000","last_clean":"2024-04-25T13:02:46.284536+0000","last_became_active":"2024-04-25T13:02:44.743032+0000","last_became_peered":"2024-04-25T13:02:44.743032+0000","last_unstale":"2024-04-25T13:02:46.284536+0000","last_undegraded":"2024-04-25T13:02:46.284536+0000","last_fullsized":"2024-04-25T13:02:46.284536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:18.403138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732563+0000","last_change":"2024-04-25T13:02:44.739403+0000","last_active":"2024-04-25T13:02:45.732563+0000","last_peered":"2024-04-25T13:02:45.732563+0000","last_clean":"2024-04-25T13:02:45.732563+0000","last_became_active":"2024-04-25T13:02:44.739097+0000","last_became_peered":"2024-04-25T13:02:44.739097+0000","last_unstale":"2024-04-25T13:02:45.732563+0000","last_undegraded":"2024-04-25T13:02:45.732563+0000","last_fullsized":"2024-04-25T13:02:45.732563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:30.519343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410693+0000","last_change":"2024-04-25T13:02:44.736821+0000","last_active":"2024-04-25T13:02:46.410693+0000","last_peered":"2024-04-25T13:02:46.410693+0000","last_clean":"2024-04-25T13:02:46.410693+0000","last_became_active":"2024-04-25T13:02:44.736099+0000","last_became_peered":"2024-04-25T13:02:44.736099+0000","last_unstale":"2024-04-25T13:02:46.410693+0000","last_undegraded":"2024-04-25T13:02:46.410693+0000","last_fullsized":"2024-04-25T13:02:46.410693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:54.858845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897609+0000","last_change":"2024-04-25T13:02:44.731035+0000","last_active":"2024-04-25T13:02:45.897609+0000","last_peered":"2024-04-25T13:02:45.897609+0000","last_clean":"2024-04-25T13:02:45.897609+0000","last_became_active":"2024-04-25T13:02:44.730717+0000","last_became_peered":"2024-04-25T13:02:44.730717+0000","last_unstale":"2024-04-25T13:02:45.897609+0000","last_undegraded":"2024-04-25T13:02:45.897609+0000","last_fullsized":"2024-04-25T13:02:45.897609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:09.061183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410231+0000","last_change":"2024-04-25T13:02:44.735851+0000","last_active":"2024-04-25T13:02:46.410231+0000","last_peered":"2024-04-25T13:02:46.410231+0000","last_clean":"2024-04-25T13:02:46.410231+0000","last_became_active":"2024-04-25T13:02:44.735559+0000","last_became_peered":"2024-04-25T13:02:44.735559+0000","last_unstale":"2024-04-25T13:02:46.410231+0000","last_undegraded":"2024-04-25T13:02:46.410231+0000","last_fullsized":"2024-04-25T13:02:46.410231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.952094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.951474+0000","last_change":"2024-04-25T13:02:43.909800+0000","last_active":"2024-04-25T13:02:48.951474+0000","last_peered":"2024-04-25T13:02:48.951474+0000","last_clean":"2024-04-25T13:02:48.951474+0000","last_became_active":"2024-04-25T13:02:43.909463+0000","last_became_peered":"2024-04-25T13:02:43.909463+0000","last_unstale":"2024-04-25T13:02:48.951474+0000","last_undegraded":"2024-04-25T13:02:48.951474+0000","last_fullsized":"2024-04-25T13:02:48.951474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:31.495586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732445+0000","last_change":"2024-04-25T13:02:44.736710+0000","last_active":"2024-04-25T13:02:45.732445+0000","last_peered":"2024-04-25T13:02:45.732445+0000","last_clean":"2024-04-25T13:02:45.732445+0000","last_became_active":"2024-04-25T13:02:44.736385+0000","last_became_peered":"2024-04-25T13:02:44.736385+0000","last_unstale":"2024-04-25T13:02:45.732445+0000","last_undegraded":"2024-04-25T13:02:45.732445+0000","last_fullsized":"2024-04-25T13:02:45.732445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.216946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732958+0000","last_change":"2024-04-25T13:02:43.918766+0000","last_active":"2024-04-25T13:02:45.732958+0000","last_peered":"2024-04-25T13:02:45.732958+0000","last_clean":"2024-04-25T13:02:45.732958+0000","last_became_active":"2024-04-25T13:02:43.918283+0000","last_became_peered":"2024-04-25T13:02:43.918283+0000","last_unstale":"2024-04-25T13:02:45.732958+0000","last_undegraded":"2024-04-25T13:02:45.732958+0000","last_fullsized":"2024-04-25T13:02:45.732958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:17.986777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284756+0000","last_change":"2024-04-25T13:02:44.735806+0000","last_active":"2024-04-25T13:02:46.284756+0000","last_peered":"2024-04-25T13:02:46.284756+0000","last_clean":"2024-04-25T13:02:46.284756+0000","last_became_active":"2024-04-25T13:02:44.735507+0000","last_became_peered":"2024-04-25T13:02:44.735507+0000","last_unstale":"2024-04-25T13:02:46.284756+0000","last_undegraded":"2024-04-25T13:02:46.284756+0000","last_fullsized":"2024-04-25T13:02:46.284756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:22.230015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.702593+0000","last_change":"2024-04-25T13:02:43.910055+0000","last_active":"2024-04-25T13:02:50.702593+0000","last_peered":"2024-04-25T13:02:50.702593+0000","last_clean":"2024-04-25T13:02:50.702593+0000","last_became_active":"2024-04-25T13:02:43.909762+0000","last_became_peered":"2024-04-25T13:02:43.909762+0000","last_unstale":"2024-04-25T13:02:50.702593+0000","last_undegraded":"2024-04-25T13:02:50.702593+0000","last_fullsized":"2024-04-25T13:02:50.702593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:47.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732985+0000","last_change":"2024-04-25T13:02:44.733253+0000","last_active":"2024-04-25T13:02:45.732985+0000","last_peered":"2024-04-25T13:02:45.732985+0000","last_clean":"2024-04-25T13:02:45.732985+0000","last_became_active":"2024-04-25T13:02:44.732917+0000","last_became_peered":"2024-04-25T13:02:44.732917+0000","last_unstale":"2024-04-25T13:02:45.732985+0000","last_undegraded":"2024-04-25T13:02:45.732985+0000","last_fullsized":"2024-04-25T13:02:45.732985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:10.380810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.700941+0000","last_change":"2024-04-25T13:02:43.901490+0000","last_active":"2024-04-25T13:02:50.700941+0000","last_peered":"2024-04-25T13:02:50.700941+0000","last_clean":"2024-04-25T13:02:50.700941+0000","last_became_active":"2024-04-25T13:02:43.900807+0000","last_became_peered":"2024-04-25T13:02:43.900807+0000","last_unstale":"2024-04-25T13:02:50.700941+0000","last_undegraded":"2024-04-25T13:02:50.700941+0000","last_fullsized":"2024-04-25T13:02:50.700941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.101575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733106+0000","last_change":"2024-04-25T13:02:44.744380+0000","last_active":"2024-04-25T13:02:45.733106+0000","last_peered":"2024-04-25T13:02:45.733106+0000","last_clean":"2024-04-25T13:02:45.733106+0000","last_became_active":"2024-04-25T13:02:44.744098+0000","last_became_peered":"2024-04-25T13:02:44.744098+0000","last_unstale":"2024-04-25T13:02:45.733106+0000","last_undegraded":"2024-04-25T13:02:45.733106+0000","last_fullsized":"2024-04-25T13:02:45.733106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:53.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732844+0000","last_change":"2024-04-25T13:02:43.908588+0000","last_active":"2024-04-25T13:02:45.732844+0000","last_peered":"2024-04-25T13:02:45.732844+0000","last_clean":"2024-04-25T13:02:45.732844+0000","last_became_active":"2024-04-25T13:02:43.908219+0000","last_became_peered":"2024-04-25T13:02:43.908219+0000","last_unstale":"2024-04-25T13:02:45.732844+0000","last_undegraded":"2024-04-25T13:02:45.732844+0000","last_fullsized":"2024-04-25T13:02:45.732844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:44.126509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897077+0000","last_change":"2024-04-25T13:02:44.737072+0000","last_active":"2024-04-25T13:02:45.897077+0000","last_peered":"2024-04-25T13:02:45.897077+0000","last_clean":"2024-04-25T13:02:45.897077+0000","last_became_active":"2024-04-25T13:02:44.736750+0000","last_became_peered":"2024-04-25T13:02:44.736750+0000","last_unstale":"2024-04-25T13:02:45.897077+0000","last_undegraded":"2024-04-25T13:02:45.897077+0000","last_fullsized":"2024-04-25T13:02:45.897077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:42.933740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950105+0000","last_change":"2024-04-25T13:02:43.913516+0000","last_active":"2024-04-25T13:02:48.950105+0000","last_peered":"2024-04-25T13:02:48.950105+0000","last_clean":"2024-04-25T13:02:48.950105+0000","last_became_active":"2024-04-25T13:02:43.913067+0000","last_became_peered":"2024-04-25T13:02:43.913067+0000","last_unstale":"2024-04-25T13:02:48.950105+0000","last_undegraded":"2024-04-25T13:02:48.950105+0000","last_fullsized":"2024-04-25T13:02:48.950105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:55.004598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732957+0000","last_change":"2024-04-25T13:02:44.741433+0000","last_active":"2024-04-25T13:02:45.732957+0000","last_peered":"2024-04-25T13:02:45.732957+0000","last_clean":"2024-04-25T13:02:45.732957+0000","last_became_active":"2024-04-25T13:02:44.741114+0000","last_became_peered":"2024-04-25T13:02:44.741114+0000","last_unstale":"2024-04-25T13:02:45.732957+0000","last_undegraded":"2024-04-25T13:02:45.732957+0000","last_fullsized":"2024-04-25T13:02:45.732957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:02.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.951955+0000","last_change":"2024-04-25T13:02:43.900699+0000","last_active":"2024-04-25T13:02:48.951955+0000","last_peered":"2024-04-25T13:02:48.951955+0000","last_clean":"2024-04-25T13:02:48.951955+0000","last_became_active":"2024-04-25T13:02:43.900306+0000","last_became_peered":"2024-04-25T13:02:43.900306+0000","last_unstale":"2024-04-25T13:02:48.951955+0000","last_undegraded":"2024-04-25T13:02:48.951955+0000","last_fullsized":"2024-04-25T13:02:48.951955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.720365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732297+0000","last_change":"2024-04-25T13:02:44.733157+0000","last_active":"2024-04-25T13:02:45.732297+0000","last_peered":"2024-04-25T13:02:45.732297+0000","last_clean":"2024-04-25T13:02:45.732297+0000","last_became_active":"2024-04-25T13:02:44.732836+0000","last_became_peered":"2024-04-25T13:02:44.732836+0000","last_unstale":"2024-04-25T13:02:45.732297+0000","last_undegraded":"2024-04-25T13:02:45.732297+0000","last_fullsized":"2024-04-25T13:02:45.732297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:30.189708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732615+0000","last_change":"2024-04-25T13:02:43.913362+0000","last_active":"2024-04-25T13:02:45.732615+0000","last_peered":"2024-04-25T13:02:45.732615+0000","last_clean":"2024-04-25T13:02:45.732615+0000","last_became_active":"2024-04-25T13:02:43.912763+0000","last_became_peered":"2024-04-25T13:02:43.912763+0000","last_unstale":"2024-04-25T13:02:45.732615+0000","last_undegraded":"2024-04-25T13:02:45.732615+0000","last_fullsized":"2024-04-25T13:02:45.732615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:12.216165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897705+0000","last_change":"2024-04-25T13:02:44.732662+0000","last_active":"2024-04-25T13:02:45.897705+0000","last_peered":"2024-04-25T13:02:45.897705+0000","last_clean":"2024-04-25T13:02:45.897705+0000","last_became_active":"2024-04-25T13:02:44.732319+0000","last_became_peered":"2024-04-25T13:02:44.732319+0000","last_unstale":"2024-04-25T13:02:45.897705+0000","last_undegraded":"2024-04-25T13:02:45.897705+0000","last_fullsized":"2024-04-25T13:02:45.897705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:51.683916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.938932+0000","last_change":"2024-04-25T13:02:43.911019+0000","last_active":"2024-04-25T13:02:47.938932+0000","last_peered":"2024-04-25T13:02:47.938932+0000","last_clean":"2024-04-25T13:02:47.938932+0000","last_became_active":"2024-04-25T13:02:43.910707+0000","last_became_peered":"2024-04-25T13:02:43.910707+0000","last_unstale":"2024-04-25T13:02:47.938932+0000","last_undegraded":"2024-04-25T13:02:47.938932+0000","last_fullsized":"2024-04-25T13:02:47.938932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:07.042580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733012+0000","last_change":"2024-04-25T13:02:44.732791+0000","last_active":"2024-04-25T13:02:45.733012+0000","last_peered":"2024-04-25T13:02:45.733012+0000","last_clean":"2024-04-25T13:02:45.733012+0000","last_became_active":"2024-04-25T13:02:44.732484+0000","last_became_peered":"2024-04-25T13:02:44.732484+0000","last_unstale":"2024-04-25T13:02:45.733012+0000","last_undegraded":"2024-04-25T13:02:45.733012+0000","last_fullsized":"2024-04-25T13:02:45.733012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:36.164774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.937394+0000","last_change":"2024-04-25T13:02:43.912447+0000","last_active":"2024-04-25T13:02:47.937394+0000","last_peered":"2024-04-25T13:02:47.937394+0000","last_clean":"2024-04-25T13:02:47.937394+0000","last_became_active":"2024-04-25T13:02:43.912091+0000","last_became_peered":"2024-04-25T13:02:43.912091+0000","last_unstale":"2024-04-25T13:02:47.937394+0000","last_undegraded":"2024-04-25T13:02:47.937394+0000","last_fullsized":"2024-04-25T13:02:47.937394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:41.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284787+0000","last_change":"2024-04-25T13:02:44.739021+0000","last_active":"2024-04-25T13:02:46.284787+0000","last_peered":"2024-04-25T13:02:46.284787+0000","last_clean":"2024-04-25T13:02:46.284787+0000","last_became_active":"2024-04-25T13:02:44.737945+0000","last_became_peered":"2024-04-25T13:02:44.737945+0000","last_unstale":"2024-04-25T13:02:46.284787+0000","last_undegraded":"2024-04-25T13:02:46.284787+0000","last_fullsized":"2024-04-25T13:02:46.284787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:09.405021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733106+0000","last_change":"2024-04-25T13:02:43.909061+0000","last_active":"2024-04-25T13:02:45.733106+0000","last_peered":"2024-04-25T13:02:45.733106+0000","last_clean":"2024-04-25T13:02:45.733106+0000","last_became_active":"2024-04-25T13:02:43.908739+0000","last_became_peered":"2024-04-25T13:02:43.908739+0000","last_unstale":"2024-04-25T13:02:45.733106+0000","last_undegraded":"2024-04-25T13:02:45.733106+0000","last_fullsized":"2024-04-25T13:02:45.733106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:25.720808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732998+0000","last_change":"2024-04-25T13:02:44.734346+0000","last_active":"2024-04-25T13:02:45.732998+0000","last_peered":"2024-04-25T13:02:45.732998+0000","last_clean":"2024-04-25T13:02:45.732998+0000","last_became_active":"2024-04-25T13:02:44.734045+0000","last_became_peered":"2024-04-25T13:02:44.734045+0000","last_unstale":"2024-04-25T13:02:45.732998+0000","last_undegraded":"2024-04-25T13:02:45.732998+0000","last_fullsized":"2024-04-25T13:02:45.732998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:42.137455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.939198+0000","last_change":"2024-04-25T13:02:43.910212+0000","last_active":"2024-04-25T13:02:47.939198+0000","last_peered":"2024-04-25T13:02:47.939198+0000","last_clean":"2024-04-25T13:02:47.939198+0000","last_became_active":"2024-04-25T13:02:43.909873+0000","last_became_peered":"2024-04-25T13:02:43.909873+0000","last_unstale":"2024-04-25T13:02:47.939198+0000","last_undegraded":"2024-04-25T13:02:47.939198+0000","last_fullsized":"2024-04-25T13:02:47.939198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.716434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897170+0000","last_change":"2024-04-25T13:02:44.731633+0000","last_active":"2024-04-25T13:02:45.897170+0000","last_peered":"2024-04-25T13:02:45.897170+0000","last_clean":"2024-04-25T13:02:45.897170+0000","last_became_active":"2024-04-25T13:02:44.731360+0000","last_became_peered":"2024-04-25T13:02:44.731360+0000","last_unstale":"2024-04-25T13:02:45.897170+0000","last_undegraded":"2024-04-25T13:02:45.897170+0000","last_fullsized":"2024-04-25T13:02:45.897170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:12.960787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.948418+0000","last_change":"2024-04-25T13:02:43.909925+0000","last_active":"2024-04-25T13:02:48.948418+0000","last_peered":"2024-04-25T13:02:48.948418+0000","last_clean":"2024-04-25T13:02:48.948418+0000","last_became_active":"2024-04-25T13:02:43.909602+0000","last_became_peered":"2024-04-25T13:02:43.909602+0000","last_unstale":"2024-04-25T13:02:48.948418+0000","last_undegraded":"2024-04-25T13:02:48.948418+0000","last_fullsized":"2024-04-25T13:02:48.948418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:09.196604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410049+0000","last_change":"2024-04-25T13:02:44.742431+0000","last_active":"2024-04-25T13:02:46.410049+0000","last_peered":"2024-04-25T13:02:46.410049+0000","last_clean":"2024-04-25T13:02:46.410049+0000","last_became_active":"2024-04-25T13:02:44.742120+0000","last_became_peered":"2024-04-25T13:02:44.742120+0000","last_unstale":"2024-04-25T13:02:46.410049+0000","last_undegraded":"2024-04-25T13:02:46.410049+0000","last_fullsized":"2024-04-25T13:02:46.410049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:08.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732789+0000","last_change":"2024-04-25T13:02:43.911342+0000","last_active":"2024-04-25T13:02:45.732789+0000","last_peered":"2024-04-25T13:02:45.732789+0000","last_clean":"2024-04-25T13:02:45.732789+0000","last_became_active":"2024-04-25T13:02:43.910741+0000","last_became_peered":"2024-04-25T13:02:43.910741+0000","last_unstale":"2024-04-25T13:02:45.732789+0000","last_undegraded":"2024-04-25T13:02:45.732789+0000","last_fullsized":"2024-04-25T13:02:45.732789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:28.464712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410923+0000","last_change":"2024-04-25T13:02:44.736639+0000","last_active":"2024-04-25T13:02:46.410923+0000","last_peered":"2024-04-25T13:02:46.410923+0000","last_clean":"2024-04-25T13:02:46.410923+0000","last_became_active":"2024-04-25T13:02:44.736285+0000","last_became_peered":"2024-04-25T13:02:44.736285+0000","last_unstale":"2024-04-25T13:02:46.410923+0000","last_undegraded":"2024-04-25T13:02:46.410923+0000","last_fullsized":"2024-04-25T13:02:46.410923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.840298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410199+0000","last_change":"2024-04-25T13:02:43.906893+0000","last_active":"2024-04-25T13:02:46.410199+0000","last_peered":"2024-04-25T13:02:46.410199+0000","last_clean":"2024-04-25T13:02:46.410199+0000","last_became_active":"2024-04-25T13:02:43.906538+0000","last_became_peered":"2024-04-25T13:02:43.906538+0000","last_unstale":"2024-04-25T13:02:46.410199+0000","last_undegraded":"2024-04-25T13:02:46.410199+0000","last_fullsized":"2024-04-25T13:02:46.410199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:10.601792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410359+0000","last_change":"2024-04-25T13:02:44.741092+0000","last_active":"2024-04-25T13:02:46.410359+0000","last_peered":"2024-04-25T13:02:46.410359+0000","last_clean":"2024-04-25T13:02:46.410359+0000","last_became_active":"2024-04-25T13:02:44.740755+0000","last_became_peered":"2024-04-25T13:02:44.740755+0000","last_unstale":"2024-04-25T13:02:46.410359+0000","last_undegraded":"2024-04-25T13:02:46.410359+0000","last_fullsized":"2024-04-25T13:02:46.410359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:49.124211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.692989+0000","last_change":"2024-04-25T13:02:43.909572+0000","last_active":"2024-04-25T13:02:50.692989+0000","last_peered":"2024-04-25T13:02:50.692989+0000","last_clean":"2024-04-25T13:02:50.692989+0000","last_became_active":"2024-04-25T13:02:43.909109+0000","last_became_peered":"2024-04-25T13:02:43.909109+0000","last_unstale":"2024-04-25T13:02:50.692989+0000","last_undegraded":"2024-04-25T13:02:50.692989+0000","last_fullsized":"2024-04-25T13:02:50.692989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.505599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732701+0000","last_change":"2024-04-25T13:02:44.740972+0000","last_active":"2024-04-25T13:02:45.732701+0000","last_peered":"2024-04-25T13:02:45.732701+0000","last_clean":"2024-04-25T13:02:45.732701+0000","last_became_active":"2024-04-25T13:02:44.740620+0000","last_became_peered":"2024-04-25T13:02:44.740620+0000","last_unstale":"2024-04-25T13:02:45.732701+0000","last_undegraded":"2024-04-25T13:02:45.732701+0000","last_fullsized":"2024-04-25T13:02:45.732701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:45.787763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.694052+0000","last_change":"2024-04-25T13:02:43.901205+0000","last_active":"2024-04-25T13:02:50.694052+0000","last_peered":"2024-04-25T13:02:50.694052+0000","last_clean":"2024-04-25T13:02:50.694052+0000","last_became_active":"2024-04-25T13:02:43.900796+0000","last_became_peered":"2024-04-25T13:02:43.900796+0000","last_unstale":"2024-04-25T13:02:50.694052+0000","last_undegraded":"2024-04-25T13:02:50.694052+0000","last_fullsized":"2024-04-25T13:02:50.694052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:14.020387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410546+0000","last_change":"2024-04-25T13:02:44.739119+0000","last_active":"2024-04-25T13:02:46.410546+0000","last_peered":"2024-04-25T13:02:46.410546+0000","last_clean":"2024-04-25T13:02:46.410546+0000","last_became_active":"2024-04-25T13:02:44.738804+0000","last_became_peered":"2024-04-25T13:02:44.738804+0000","last_unstale":"2024-04-25T13:02:46.410546+0000","last_undegraded":"2024-04-25T13:02:46.410546+0000","last_fullsized":"2024-04-25T13:02:46.410546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:28.424093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.897505+0000","last_change":"2024-04-25T13:02:43.911812+0000","last_active":"2024-04-25T13:02:45.897505+0000","last_peered":"2024-04-25T13:02:45.897505+0000","last_clean":"2024-04-25T13:02:45.897505+0000","last_became_active":"2024-04-25T13:02:43.911404+0000","last_became_peered":"2024-04-25T13:02:43.911404+0000","last_unstale":"2024-04-25T13:02:45.897505+0000","last_undegraded":"2024-04-25T13:02:45.897505+0000","last_fullsized":"2024-04-25T13:02:45.897505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:17.210980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036573+0000","last_change":"2024-04-25T13:02:44.732546+0000","last_active":"2024-04-25T13:02:46.036573+0000","last_peered":"2024-04-25T13:02:46.036573+0000","last_clean":"2024-04-25T13:02:46.036573+0000","last_became_active":"2024-04-25T13:02:44.732238+0000","last_became_peered":"2024-04-25T13:02:44.732238+0000","last_unstale":"2024-04-25T13:02:46.036573+0000","last_undegraded":"2024-04-25T13:02:46.036573+0000","last_fullsized":"2024-04-25T13:02:46.036573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.293939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410692+0000","last_change":"2024-04-25T13:02:43.910620+0000","last_active":"2024-04-25T13:02:46.410692+0000","last_peered":"2024-04-25T13:02:46.410692+0000","last_clean":"2024-04-25T13:02:46.410692+0000","last_became_active":"2024-04-25T13:02:43.910325+0000","last_became_peered":"2024-04-25T13:02:43.910325+0000","last_unstale":"2024-04-25T13:02:46.410692+0000","last_undegraded":"2024-04-25T13:02:46.410692+0000","last_fullsized":"2024-04-25T13:02:46.410692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.185209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.036326+0000","last_change":"2024-04-25T13:02:44.737134+0000","last_active":"2024-04-25T13:02:46.036326+0000","last_peered":"2024-04-25T13:02:46.036326+0000","last_clean":"2024-04-25T13:02:46.036326+0000","last_became_active":"2024-04-25T13:02:44.736790+0000","last_became_peered":"2024-04-25T13:02:44.736790+0000","last_unstale":"2024-04-25T13:02:46.036326+0000","last_undegraded":"2024-04-25T13:02:46.036326+0000","last_fullsized":"2024-04-25T13:02:46.036326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:02.680752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732933+0000","last_change":"2024-04-25T13:02:44.740743+0000","last_active":"2024-04-25T13:02:45.732933+0000","last_peered":"2024-04-25T13:02:45.732933+0000","last_clean":"2024-04-25T13:02:45.732933+0000","last_became_active":"2024-04-25T13:02:44.740421+0000","last_became_peered":"2024-04-25T13:02:44.740421+0000","last_unstale":"2024-04-25T13:02:45.732933+0000","last_undegraded":"2024-04-25T13:02:45.732933+0000","last_fullsized":"2024-04-25T13:02:45.732933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950953+0000","last_change":"2024-04-25T13:02:43.920678+0000","last_active":"2024-04-25T13:02:48.950953+0000","last_peered":"2024-04-25T13:02:48.950953+0000","last_clean":"2024-04-25T13:02:48.950953+0000","last_became_active":"2024-04-25T13:02:43.920338+0000","last_became_peered":"2024-04-25T13:02:43.920338+0000","last_unstale":"2024-04-25T13:02:48.950953+0000","last_undegraded":"2024-04-25T13:02:48.950953+0000","last_fullsized":"2024-04-25T13:02:48.950953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732583+0000","last_change":"2024-04-25T13:02:44.736655+0000","last_active":"2024-04-25T13:02:45.732583+0000","last_peered":"2024-04-25T13:02:45.732583+0000","last_clean":"2024-04-25T13:02:45.732583+0000","last_became_active":"2024-04-25T13:02:44.736340+0000","last_became_peered":"2024-04-25T13:02:44.736340+0000","last_unstale":"2024-04-25T13:02:45.732583+0000","last_undegraded":"2024-04-25T13:02:45.732583+0000","last_fullsized":"2024-04-25T13:02:45.732583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:35.129606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410403+0000","last_change":"2024-04-25T13:02:43.903179+0000","last_active":"2024-04-25T13:02:46.410403+0000","last_peered":"2024-04-25T13:02:46.410403+0000","last_clean":"2024-04-25T13:02:46.410403+0000","last_became_active":"2024-04-25T13:02:43.902799+0000","last_became_peered":"2024-04-25T13:02:43.902799+0000","last_unstale":"2024-04-25T13:02:46.410403+0000","last_undegraded":"2024-04-25T13:02:46.410403+0000","last_fullsized":"2024-04-25T13:02:46.410403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.472772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410553+0000","last_change":"2024-04-25T13:01:33.625244+0000","last_active":"2024-04-25T13:02:46.410553+0000","last_peered":"2024-04-25T13:02:46.410553+0000","last_clean":"2024-04-25T13:02:46.410553+0000","last_became_active":"2024-04-25T13:01:33.624760+0000","last_became_peered":"2024-04-25T13:01:33.624760+0000","last_unstale":"2024-04-25T13:02:46.410553+0000","last_undegraded":"2024-04-25T13:02:46.410553+0000","last_fullsized":"2024-04-25T13:02:46.410553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_clean_scrub_stamp":"2024-04-25T12:58:01.939677+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:21.636432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.409923+0000","last_change":"2024-04-25T13:02:44.732711+0000","last_active":"2024-04-25T13:02:46.409923+0000","last_peered":"2024-04-25T13:02:46.409923+0000","last_clean":"2024-04-25T13:02:46.409923+0000","last_became_active":"2024-04-25T13:02:44.732394+0000","last_became_peered":"2024-04-25T13:02:44.732394+0000","last_unstale":"2024-04-25T13:02:46.409923+0000","last_undegraded":"2024-04-25T13:02:46.409923+0000","last_fullsized":"2024-04-25T13:02:46.409923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:02.601328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950449+0000","last_change":"2024-04-25T13:02:43.908773+0000","last_active":"2024-04-25T13:02:48.950449+0000","last_peered":"2024-04-25T13:02:48.950449+0000","last_clean":"2024-04-25T13:02:48.950449+0000","last_became_active":"2024-04-25T13:02:43.908440+0000","last_became_peered":"2024-04-25T13:02:43.908440+0000","last_unstale":"2024-04-25T13:02:48.950449+0000","last_undegraded":"2024-04-25T13:02:48.950449+0000","last_fullsized":"2024-04-25T13:02:48.950449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:49.565391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:50.693524+0000","last_change":"2024-04-25T13:02:43.908686+0000","last_active":"2024-04-25T13:02:50.693524+0000","last_peered":"2024-04-25T13:02:50.693524+0000","last_clean":"2024-04-25T13:02:50.693524+0000","last_became_active":"2024-04-25T13:02:43.908391+0000","last_became_peered":"2024-04-25T13:02:43.908391+0000","last_unstale":"2024-04-25T13:02:50.693524+0000","last_undegraded":"2024-04-25T13:02:50.693524+0000","last_fullsized":"2024-04-25T13:02:50.693524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:49.938946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410244+0000","last_change":"2024-04-25T13:02:44.730629+0000","last_active":"2024-04-25T13:02:46.410244+0000","last_peered":"2024-04-25T13:02:46.410244+0000","last_clean":"2024-04-25T13:02:46.410244+0000","last_became_active":"2024-04-25T13:02:44.730288+0000","last_became_peered":"2024-04-25T13:02:44.730288+0000","last_unstale":"2024-04-25T13:02:46.410244+0000","last_undegraded":"2024-04-25T13:02:46.410244+0000","last_fullsized":"2024-04-25T13:02:46.410244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:26.255183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.950741+0000","last_change":"2024-04-25T13:02:43.918791+0000","last_active":"2024-04-25T13:02:48.950741+0000","last_peered":"2024-04-25T13:02:48.950741+0000","last_clean":"2024-04-25T13:02:48.950741+0000","last_became_active":"2024-04-25T13:02:43.918454+0000","last_became_peered":"2024-04-25T13:02:43.918454+0000","last_unstale":"2024-04-25T13:02:48.950741+0000","last_undegraded":"2024-04-25T13:02:48.950741+0000","last_fullsized":"2024-04-25T13:02:48.950741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.137235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.733010+0000","last_change":"2024-04-25T13:02:44.740324+0000","last_active":"2024-04-25T13:02:45.733010+0000","last_peered":"2024-04-25T13:02:45.733010+0000","last_clean":"2024-04-25T13:02:45.733010+0000","last_became_active":"2024-04-25T13:02:44.739993+0000","last_became_peered":"2024-04-25T13:02:44.739993+0000","last_unstale":"2024-04-25T13:02:45.733010+0000","last_undegraded":"2024-04-25T13:02:45.733010+0000","last_fullsized":"2024-04-25T13:02:45.733010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.207392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410228+0000","last_change":"2024-04-25T13:02:43.903356+0000","last_active":"2024-04-25T13:02:46.410228+0000","last_peered":"2024-04-25T13:02:46.410228+0000","last_clean":"2024-04-25T13:02:46.410228+0000","last_became_active":"2024-04-25T13:02:43.903046+0000","last_became_peered":"2024-04-25T13:02:43.903046+0000","last_unstale":"2024-04-25T13:02:46.410228+0000","last_undegraded":"2024-04-25T13:02:46.410228+0000","last_fullsized":"2024-04-25T13:02:46.410228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.514373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284851+0000","last_change":"2024-04-25T13:02:44.745033+0000","last_active":"2024-04-25T13:02:46.284851+0000","last_peered":"2024-04-25T13:02:46.284851+0000","last_clean":"2024-04-25T13:02:46.284851+0000","last_became_active":"2024-04-25T13:02:44.744718+0000","last_became_peered":"2024-04-25T13:02:44.744718+0000","last_unstale":"2024-04-25T13:02:46.284851+0000","last_undegraded":"2024-04-25T13:02:46.284851+0000","last_fullsized":"2024-04-25T13:02:46.284851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:26.736807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732494+0000","last_change":"2024-04-25T13:02:43.909366+0000","last_active":"2024-04-25T13:02:45.732494+0000","last_peered":"2024-04-25T13:02:45.732494+0000","last_clean":"2024-04-25T13:02:45.732494+0000","last_became_active":"2024-04-25T13:02:43.908968+0000","last_became_peered":"2024-04-25T13:02:43.908968+0000","last_unstale":"2024-04-25T13:02:45.732494+0000","last_undegraded":"2024-04-25T13:02:45.732494+0000","last_fullsized":"2024-04-25T13:02:45.732494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:36.915466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.896963+0000","last_change":"2024-04-25T13:02:44.730481+0000","last_active":"2024-04-25T13:02:45.896963+0000","last_peered":"2024-04-25T13:02:45.896963+0000","last_clean":"2024-04-25T13:02:45.896963+0000","last_became_active":"2024-04-25T13:02:44.730163+0000","last_became_peered":"2024-04-25T13:02:44.730163+0000","last_unstale":"2024-04-25T13:02:45.896963+0000","last_undegraded":"2024-04-25T13:02:45.896963+0000","last_fullsized":"2024-04-25T13:02:45.896963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:12.207840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:49.968452+0000","last_change":"2024-04-25T13:02:43.922262+0000","last_active":"2024-04-25T13:02:49.968452+0000","last_peered":"2024-04-25T13:02:49.968452+0000","last_clean":"2024-04-25T13:02:49.968452+0000","last_became_active":"2024-04-25T13:02:43.921356+0000","last_became_peered":"2024-04-25T13:02:43.921356+0000","last_unstale":"2024-04-25T13:02:49.968452+0000","last_undegraded":"2024-04-25T13:02:49.968452+0000","last_fullsized":"2024-04-25T13:02:49.968452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:27.446112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410083+0000","last_change":"2024-04-25T13:02:44.735879+0000","last_active":"2024-04-25T13:02:46.410083+0000","last_peered":"2024-04-25T13:02:46.410083+0000","last_clean":"2024-04-25T13:02:46.410083+0000","last_became_active":"2024-04-25T13:02:44.735574+0000","last_became_peered":"2024-04-25T13:02:44.735574+0000","last_unstale":"2024-04-25T13:02:46.410083+0000","last_undegraded":"2024-04-25T13:02:46.410083+0000","last_fullsized":"2024-04-25T13:02:46.410083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:14.094813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410077+0000","last_change":"2024-04-25T13:02:43.912250+0000","last_active":"2024-04-25T13:02:46.410077+0000","last_peered":"2024-04-25T13:02:46.410077+0000","last_clean":"2024-04-25T13:02:46.410077+0000","last_became_active":"2024-04-25T13:02:43.911945+0000","last_became_peered":"2024-04-25T13:02:43.911945+0000","last_unstale":"2024-04-25T13:02:46.410077+0000","last_undegraded":"2024-04-25T13:02:46.410077+0000","last_fullsized":"2024-04-25T13:02:46.410077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:51.872195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732542+0000","last_change":"2024-04-25T13:02:44.739184+0000","last_active":"2024-04-25T13:02:45.732542+0000","last_peered":"2024-04-25T13:02:45.732542+0000","last_clean":"2024-04-25T13:02:45.732542+0000","last_became_active":"2024-04-25T13:02:44.738862+0000","last_became_peered":"2024-04-25T13:02:44.738862+0000","last_unstale":"2024-04-25T13:02:45.732542+0000","last_undegraded":"2024-04-25T13:02:45.732542+0000","last_fullsized":"2024-04-25T13:02:45.732542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.605730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:47.940196+0000","last_change":"2024-04-25T13:02:43.910735+0000","last_active":"2024-04-25T13:02:47.940196+0000","last_peered":"2024-04-25T13:02:47.940196+0000","last_clean":"2024-04-25T13:02:47.940196+0000","last_became_active":"2024-04-25T13:02:43.910110+0000","last_became_peered":"2024-04-25T13:02:43.910110+0000","last_unstale":"2024-04-25T13:02:47.940196+0000","last_undegraded":"2024-04-25T13:02:47.940196+0000","last_fullsized":"2024-04-25T13:02:47.940196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:49.114560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410957+0000","last_change":"2024-04-25T13:02:44.738410+0000","last_active":"2024-04-25T13:02:46.410957+0000","last_peered":"2024-04-25T13:02:46.410957+0000","last_clean":"2024-04-25T13:02:46.410957+0000","last_became_active":"2024-04-25T13:02:44.738091+0000","last_became_peered":"2024-04-25T13:02:44.738091+0000","last_unstale":"2024-04-25T13:02:46.410957+0000","last_undegraded":"2024-04-25T13:02:46.410957+0000","last_fullsized":"2024-04-25T13:02:46.410957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.737907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:48.952272+0000","last_change":"2024-04-25T13:02:43.914563+0000","last_active":"2024-04-25T13:02:48.952272+0000","last_peered":"2024-04-25T13:02:48.952272+0000","last_clean":"2024-04-25T13:02:48.952272+0000","last_became_active":"2024-04-25T13:02:43.914274+0000","last_became_peered":"2024-04-25T13:02:43.914274+0000","last_unstale":"2024-04-25T13:02:48.952272+0000","last_undegraded":"2024-04-25T13:02:48.952272+0000","last_fullsized":"2024-04-25T13:02:48.952272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.411040+0000","last_change":"2024-04-25T13:02:44.730098+0000","last_active":"2024-04-25T13:02:46.411040+0000","last_peered":"2024-04-25T13:02:46.411040+0000","last_clean":"2024-04-25T13:02:46.411040+0000","last_became_active":"2024-04-25T13:02:44.729656+0000","last_became_peered":"2024-04-25T13:02:44.729656+0000","last_unstale":"2024-04-25T13:02:46.411040+0000","last_undegraded":"2024-04-25T13:02:46.411040+0000","last_fullsized":"2024-04-25T13:02:46.411040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.344967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732984+0000","last_change":"2024-04-25T13:02:43.911593+0000","last_active":"2024-04-25T13:02:45.732984+0000","last_peered":"2024-04-25T13:02:45.732984+0000","last_clean":"2024-04-25T13:02:45.732984+0000","last_became_active":"2024-04-25T13:02:43.910484+0000","last_became_peered":"2024-04-25T13:02:43.910484+0000","last_unstale":"2024-04-25T13:02:45.732984+0000","last_undegraded":"2024-04-25T13:02:45.732984+0000","last_fullsized":"2024-04-25T13:02:45.732984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:52.257269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410898+0000","last_change":"2024-04-25T13:02:44.737141+0000","last_active":"2024-04-25T13:02:46.410898+0000","last_peered":"2024-04-25T13:02:46.410898+0000","last_clean":"2024-04-25T13:02:46.410898+0000","last_became_active":"2024-04-25T13:02:44.736464+0000","last_became_peered":"2024-04-25T13:02:44.736464+0000","last_unstale":"2024-04-25T13:02:46.410898+0000","last_undegraded":"2024-04-25T13:02:46.410898+0000","last_fullsized":"2024-04-25T13:02:46.410898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:16.337344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410217+0000","last_change":"2024-04-25T13:02:43.921129+0000","last_active":"2024-04-25T13:02:46.410217+0000","last_peered":"2024-04-25T13:02:46.410217+0000","last_clean":"2024-04-25T13:02:46.410217+0000","last_became_active":"2024-04-25T13:02:43.920768+0000","last_became_peered":"2024-04-25T13:02:43.920768+0000","last_unstale":"2024-04-25T13:02:46.410217+0000","last_undegraded":"2024-04-25T13:02:46.410217+0000","last_fullsized":"2024-04-25T13:02:46.410217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:49.178313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.411014+0000","last_change":"2024-04-25T13:02:44.731934+0000","last_active":"2024-04-25T13:02:46.411014+0000","last_peered":"2024-04-25T13:02:46.411014+0000","last_clean":"2024-04-25T13:02:46.411014+0000","last_became_active":"2024-04-25T13:02:44.731624+0000","last_became_peered":"2024-04-25T13:02:44.731624+0000","last_unstale":"2024-04-25T13:02:46.411014+0000","last_undegraded":"2024-04-25T13:02:46.411014+0000","last_fullsized":"2024-04-25T13:02:46.411014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.809419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410087+0000","last_change":"2024-04-25T13:02:44.736982+0000","last_active":"2024-04-25T13:02:46.410087+0000","last_peered":"2024-04-25T13:02:46.410087+0000","last_clean":"2024-04-25T13:02:46.410087+0000","last_became_active":"2024-04-25T13:02:44.735127+0000","last_became_peered":"2024-04-25T13:02:44.735127+0000","last_unstale":"2024-04-25T13:02:46.410087+0000","last_undegraded":"2024-04-25T13:02:46.410087+0000","last_fullsized":"2024-04-25T13:02:46.410087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:06.827686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.409910+0000","last_change":"2024-04-25T13:02:44.737994+0000","last_active":"2024-04-25T13:02:46.409910+0000","last_peered":"2024-04-25T13:02:46.409910+0000","last_clean":"2024-04-25T13:02:46.409910+0000","last_became_active":"2024-04-25T13:02:44.737439+0000","last_became_peered":"2024-04-25T13:02:44.737439+0000","last_unstale":"2024-04-25T13:02:46.409910+0000","last_undegraded":"2024-04-25T13:02:46.409910+0000","last_fullsized":"2024-04-25T13:02:46.409910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:53.755757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410151+0000","last_change":"2024-04-25T13:02:44.739660+0000","last_active":"2024-04-25T13:02:46.410151+0000","last_peered":"2024-04-25T13:02:46.410151+0000","last_clean":"2024-04-25T13:02:46.410151+0000","last_became_active":"2024-04-25T13:02:44.738553+0000","last_became_peered":"2024-04-25T13:02:44.738553+0000","last_unstale":"2024-04-25T13:02:46.410151+0000","last_undegraded":"2024-04-25T13:02:46.410151+0000","last_fullsized":"2024-04-25T13:02:46.410151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:07.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.284578+0000","last_change":"2024-04-25T13:02:44.736644+0000","last_active":"2024-04-25T13:02:46.284578+0000","last_peered":"2024-04-25T13:02:46.284578+0000","last_clean":"2024-04-25T13:02:46.284578+0000","last_became_active":"2024-04-25T13:02:44.736325+0000","last_became_peered":"2024-04-25T13:02:44.736325+0000","last_unstale":"2024-04-25T13:02:46.284578+0000","last_undegraded":"2024-04-25T13:02:46.284578+0000","last_fullsized":"2024-04-25T13:02:46.284578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:34.102364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410279+0000","last_change":"2024-04-25T13:02:44.739324+0000","last_active":"2024-04-25T13:02:46.410279+0000","last_peered":"2024-04-25T13:02:46.410279+0000","last_clean":"2024-04-25T13:02:46.410279+0000","last_became_active":"2024-04-25T13:02:44.738994+0000","last_became_peered":"2024-04-25T13:02:44.738994+0000","last_unstale":"2024-04-25T13:02:46.410279+0000","last_undegraded":"2024-04-25T13:02:46.410279+0000","last_fullsized":"2024-04-25T13:02:46.410279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:50.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410481+0000","last_change":"2024-04-25T13:02:44.738075+0000","last_active":"2024-04-25T13:02:46.410481+0000","last_peered":"2024-04-25T13:02:46.410481+0000","last_clean":"2024-04-25T13:02:46.410481+0000","last_became_active":"2024-04-25T13:02:44.736616+0000","last_became_peered":"2024-04-25T13:02:44.736616+0000","last_unstale":"2024-04-25T13:02:46.410481+0000","last_undegraded":"2024-04-25T13:02:46.410481+0000","last_fullsized":"2024-04-25T13:02:46.410481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:14.875245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410401+0000","last_change":"2024-04-25T13:02:44.730926+0000","last_active":"2024-04-25T13:02:46.410401+0000","last_peered":"2024-04-25T13:02:46.410401+0000","last_clean":"2024-04-25T13:02:46.410401+0000","last_became_active":"2024-04-25T13:02:44.730154+0000","last_became_peered":"2024-04-25T13:02:44.730154+0000","last_unstale":"2024-04-25T13:02:46.410401+0000","last_undegraded":"2024-04-25T13:02:46.410401+0000","last_fullsized":"2024-04-25T13:02:46.410401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:07.361869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732990+0000","last_change":"2024-04-25T13:02:44.740104+0000","last_active":"2024-04-25T13:02:45.732990+0000","last_peered":"2024-04-25T13:02:45.732990+0000","last_clean":"2024-04-25T13:02:45.732990+0000","last_became_active":"2024-04-25T13:02:44.739570+0000","last_became_peered":"2024-04-25T13:02:44.739570+0000","last_unstale":"2024-04-25T13:02:45.732990+0000","last_undegraded":"2024-04-25T13:02:45.732990+0000","last_fullsized":"2024-04-25T13:02:45.732990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:25.862867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732730+0000","last_change":"2024-04-25T13:02:44.735555+0000","last_active":"2024-04-25T13:02:45.732730+0000","last_peered":"2024-04-25T13:02:45.732730+0000","last_clean":"2024-04-25T13:02:45.732730+0000","last_became_active":"2024-04-25T13:02:44.735238+0000","last_became_peered":"2024-04-25T13:02:44.735238+0000","last_unstale":"2024-04-25T13:02:45.732730+0000","last_undegraded":"2024-04-25T13:02:45.732730+0000","last_fullsized":"2024-04-25T13:02:45.732730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.611943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732283+0000","last_change":"2024-04-25T13:02:44.730959+0000","last_active":"2024-04-25T13:02:45.732283+0000","last_peered":"2024-04-25T13:02:45.732283+0000","last_clean":"2024-04-25T13:02:45.732283+0000","last_became_active":"2024-04-25T13:02:44.730583+0000","last_became_peered":"2024-04-25T13:02:44.730583+0000","last_unstale":"2024-04-25T13:02:45.732283+0000","last_undegraded":"2024-04-25T13:02:45.732283+0000","last_fullsized":"2024-04-25T13:02:45.732283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.527837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:45.732728+0000","last_change":"2024-04-25T13:02:44.734174+0000","last_active":"2024-04-25T13:02:45.732728+0000","last_peered":"2024-04-25T13:02:45.732728+0000","last_clean":"2024-04-25T13:02:45.732728+0000","last_became_active":"2024-04-25T13:02:44.733785+0000","last_became_peered":"2024-04-25T13:02:44.733785+0000","last_unstale":"2024-04-25T13:02:45.732728+0000","last_undegraded":"2024-04-25T13:02:45.732728+0000","last_fullsized":"2024-04-25T13:02:45.732728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:39.641206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410278+0000","last_change":"2024-04-25T13:02:44.740058+0000","last_active":"2024-04-25T13:02:46.410278+0000","last_peered":"2024-04-25T13:02:46.410278+0000","last_clean":"2024-04-25T13:02:46.410278+0000","last_became_active":"2024-04-25T13:02:44.739716+0000","last_became_peered":"2024-04-25T13:02:44.739716+0000","last_unstale":"2024-04-25T13:02:46.410278+0000","last_undegraded":"2024-04-25T13:02:46.410278+0000","last_fullsized":"2024-04-25T13:02:46.410278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:21.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T13:02:46.410318+0000","last_change":"2024-04-25T13:02:44.731335+0000","last_active":"2024-04-25T13:02:46.410318+0000","last_peered":"2024-04-25T13:02:46.410318+0000","last_clean":"2024-04-25T13:02:46.410318+0000","last_became_active":"2024-04-25T13:02:44.731017+0000","last_became_peered":"2024-04-25T13:02:44.731017+0000","last_unstale":"2024-04-25T13:02:46.410318+0000","last_undegraded":"2024-04-25T13:02:46.410318+0000","last_fullsized":"2024-04-25T13:02:46.410318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:29.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":288217,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4068,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":128,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":13338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1034404,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":444534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":441980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":1032122,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:02:53.148 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:02:53.148 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:02:53.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:02:53.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:52 smithi178 ceph-mon[33233]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 969 B/s rd, 3.9 KiB/s wr, 18 op/s 2024-04-25T13:02:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:52 smithi178 ceph-mon[33233]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:52 smithi178 ceph-mon[33233]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:52 smithi178 ceph-mon[33233]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:02:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:52 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/439836554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:52 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3947790792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:53.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:52 smithi104 ceph-mon[32726]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 969 B/s rd, 3.9 KiB/s wr, 18 op/s 2024-04-25T13:02:53.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:52 smithi104 ceph-mon[32726]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:52 smithi104 ceph-mon[32726]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:02:53.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:52 smithi104 ceph-mon[32726]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:02:53.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:52 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/439836554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:02:53.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:52 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3947790792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:02:53.427 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:54.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:53 smithi178 ceph-mon[33233]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:54.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:53 smithi178 ceph-mon[33233]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:54.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:53 smithi038 ceph-mon[29400]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:54.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:53 smithi038 ceph-mon[29400]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:54.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:53 smithi104 ceph-mon[32726]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:54.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:53 smithi104 ceph-mon[32726]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:02:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 -- 172.21.15.38:0/2182691966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c1019c0 msgr2=0x7f643c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 --2- 172.21.15.38:0/2182691966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c1019c0 0x7f643c101de0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f64380099c0 tx=0x7f6438009cd0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.233 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:54 smithi038 ceph-mon[29400]: pgmap v295: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5.8 KiB/s wr, 21 op/s 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 -- 172.21.15.38:0/2182691966 shutdown_connections 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 --2- 172.21.15.38:0/2182691966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f643c103560 0x7f643c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 --2- 172.21.15.38:0/2182691966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c1019c0 0x7f643c101de0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 --2- 172.21.15.38:0/2182691966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f643c102bc0 0x7f643c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 -- 172.21.15.38:0/2182691966 >> 172.21.15.38:0/2182691966 conn(0x7f643c0fcf60 msgr2=0x7f643c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 -- 172.21.15.38:0/2182691966 shutdown_connections 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.234+0000 7f6443b50700 1 -- 172.21.15.38:0/2182691966 wait complete. 2024-04-25T13:02:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 Processor -- start 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 -- start start 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f643c1019c0 0x7f643c06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f643c102bc0 0x7f643c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 0x7f643c19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d5f0 con 0x7f643c103560 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d470 con 0x7f643c102bc0 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f6443b50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d770 con 0x7f643c1019c0 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.235+0000 7f64420ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 0x7f643c19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64418ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f643c1019c0 0x7f643c06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 0x7f643c19ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37678/0 (socket says 172.21.15.38:37678) 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64410eb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f643c102bc0 0x7f643c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 -- 172.21.15.38:0/1478131427 learned_addr learned my addr 172.21.15.38:0/1478131427 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 -- 172.21.15.38:0/1478131427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f643c1019c0 msgr2=0x7f643c06b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f643c1019c0 0x7f643c06b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 -- 172.21.15.38:0/1478131427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f643c102bc0 msgr2=0x7f643c19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f643c102bc0 0x7f643c19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 -- 172.21.15.38:0/1478131427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c1a08a0 con 0x7f643c103560 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f64420ed700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 0x7f643c19ffd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f643400b970 tx=0x7f643400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643400d860 con 0x7f643c103560 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c1a0b90 con 0x7f643c103560 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f643400e430 con 0x7f643c103560 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434017660 con 0x7f643c103560 2024-04-25T13:02:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.236+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a1080 con 0x7f643c103560 2024-04-25T13:02:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.237+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c050f80 con 0x7f643c103560 2024-04-25T13:02:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.239+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f643400e5e0 con 0x7f643c103560 2024-04-25T13:02:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.239+0000 7f6432ffd700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64280a3d80 0x7f64280a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.239+0000 7f64418ec700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64280a3d80 0x7f64280a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.239+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f64340d03c0 con 0x7f643c103560 2024-04-25T13:02:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.240+0000 7f64418ec700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64280a3d80 0x7f64280a6240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6438006010 tx=0x7f6438009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:55.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:54 smithi178 ceph-mon[33233]: pgmap v295: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5.8 KiB/s wr, 21 op/s 2024-04-25T13:02:55.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.243+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6434013030 con 0x7f643c103560 2024-04-25T13:02:55.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:54 smithi104 ceph-mon[32726]: pgmap v295: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5.8 KiB/s wr, 21 op/s 2024-04-25T13:02:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.538+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f643c02e050 con 0x7f643c103560 2024-04-25T13:02:55.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.545+0000 7f6432ffd700 1 -- 172.21.15.38:0/1478131427 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f643409cdf0 con 0x7f643c103560 2024-04-25T13:02:55.543 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T13:02:55.543 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQB/VCpmljovIBAAvfZ8lowo3IhEvf70sm6Zbg== 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64280a3d80 msgr2=0x7f64280a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64280a3d80 0x7f64280a6240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6438006010 tx=0x7f6438009f10 comp rx=0 tx=0).stop 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 msgr2=0x7f643c19ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 0x7f643c19ffd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f643400b970 tx=0x7f643400bc80 comp rx=0 tx=0).stop 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 shutdown_connections 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f64280a3d80 0x7f64280a6240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f643c103560 0x7f643c19ffd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f643c1019c0 0x7f643c06b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 --2- 172.21.15.38:0/1478131427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f643c102bc0 0x7f643c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:55.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.547+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 >> 172.21.15.38:0/1478131427 conn(0x7f643c0fcf60 msgr2=0x7f643c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:55.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.548+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 shutdown_connections 2024-04-25T13:02:55.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:55.548+0000 7f6443b50700 1 -- 172.21.15.38:0/1478131427 wait complete. 2024-04-25T13:02:56.179 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:55 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1478131427' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:02:56.179 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:55 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1478131427' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:02:56.213 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T13:02:56.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:02:56.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:02:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:55 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1478131427' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:02:56.238 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:55 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1478131427' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:02:56.254 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:02:56.254 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:02:56.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:02:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:55 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1478131427' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:02:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:55 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1478131427' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:02:56.543 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:02:57.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:56 smithi178 ceph-mon[33233]: pgmap v296: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.5 KiB/s wr, 26 op/s 2024-04-25T13:02:57.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:56 smithi038 ceph-mon[29400]: pgmap v296: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.5 KiB/s wr, 26 op/s 2024-04-25T13:02:57.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:56 smithi104 ceph-mon[32726]: pgmap v296: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.5 KiB/s wr, 26 op/s 2024-04-25T13:02:58.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.215+0000 7fdd2fc16700 1 -- 172.21.15.38:0/420188965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28075590 msgr2=0x7fdd2810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.215+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/420188965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28075590 0x7fdd2810ef00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009990 tx=0x7fdd18009ca0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 -- 172.21.15.38:0/420188965 shutdown_connections 2024-04-25T13:02:58.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/420188965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 0x7fdd28111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/420188965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28074c70 0x7fdd28075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/420188965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28075590 0x7fdd2810ef00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 -- 172.21.15.38:0/420188965 >> 172.21.15.38:0/420188965 conn(0x7fdd28079ba0 msgr2=0x7fdd28079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 -- 172.21.15.38:0/420188965 shutdown_connections 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 -- 172.21.15.38:0/420188965 wait complete. 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 Processor -- start 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 -- start start 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 0x7fdd2819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.216+0000 7fdd2fc16700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28075590 0x7fdd2819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2fc16700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 0x7fdd2819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2fc16700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28114f40 con 0x7fdd28074c70 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2fc16700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28114dc0 con 0x7fdd2810f440 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2fc16700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd281150c0 con 0x7fdd28075590 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 0x7fdd2819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 0x7fdd2819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37694/0 (socket says 172.21.15.38:37694) 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 -- 172.21.15.38:0/3753781259 learned_addr learned my addr 172.21.15.38:0/3753781259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d1b1700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28075590 0x7fdd2819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2e1b3700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 0x7fdd2819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 -- 172.21.15.38:0/3753781259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28075590 msgr2=0x7fdd2819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28075590 0x7fdd2819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 -- 172.21.15.38:0/3753781259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 msgr2=0x7fdd2819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 0x7fdd2819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 -- 172.21.15.38:0/3753781259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd281a03c0 con 0x7fdd28074c70 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2e1b3700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 0x7fdd2819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2d9b2700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 0x7fdd2819b3b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e9a0 tx=0x7fdd2400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400cbd0 con 0x7fdd28074c70 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd2400cd70 con 0x7fdd28074c70 2024-04-25T13:02:58.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd281a06b0 con 0x7fdd28074c70 2024-04-25T13:02:58.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd240187a0 con 0x7fdd28074c70 2024-04-25T13:02:58.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.217+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281a7f30 con 0x7fdd28074c70 2024-04-25T13:02:58.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.218+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd28069730 con 0x7fdd28074c70 2024-04-25T13:02:58.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.219+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdd24010a20 con 0x7fdd28074c70 2024-04-25T13:02:58.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.220+0000 7fdd1effd700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd140a3cd0 0x7fdd140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:02:58.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.220+0000 7fdd2d1b1700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd140a3cd0 0x7fdd140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:02:58.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.220+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdd240d06d0 con 0x7fdd28074c70 2024-04-25T13:02:58.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.221+0000 7fdd2d1b1700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd140a3cd0 0x7fdd140a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b790 tx=0x7fdd1801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:02:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.225+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd2409d100 con 0x7fdd28074c70 2024-04-25T13:02:58.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.495+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdd2802e0a0 con 0x7fdd28074c70 2024-04-25T13:02:58.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.501+0000 7fdd1effd700 1 -- 172.21.15.38:0/3753781259 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7fdd2409ca00 con 0x7fdd28074c70 2024-04-25T13:02:58.503 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd140a3cd0 msgr2=0x7fdd140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd140a3cd0 0x7fdd140a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b790 tx=0x7fdd1801a040 comp rx=0 tx=0).stop 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 msgr2=0x7fdd2819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 0x7fdd2819b3b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e9a0 tx=0x7fdd2400ed60 comp rx=0 tx=0).stop 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 shutdown_connections 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdd140a3cd0 0x7fdd140a6190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2810f440 0x7fdd2819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd28074c70 0x7fdd2819b3b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 --2- 172.21.15.38:0/3753781259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28075590 0x7fdd2819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 >> 172.21.15.38:0/3753781259 conn(0x7fdd28079ba0 msgr2=0x7fdd280ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 shutdown_connections 2024-04-25T13:02:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:02:58.504+0000 7fdd2fc16700 1 -- 172.21.15.38:0/3753781259 wait complete. 2024-04-25T13:02:59.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:58 smithi038 ceph-mon[29400]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-04-25T13:02:59.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:02:58 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3753781259' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:02:59.284 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1333793113},{"type":"v1","addr":"172.21.15.38:6801","nonce":1333793113}]},"active_addr":"172.21.15.38:6801/1333793113","active_change":"2024-04-25T12:55:35.192181+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.38:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":448455346}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1475454542}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1381358988}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":3705953772}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2537726626}]}]} 2024-04-25T13:02:59.291 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:02:59.291 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:02:59.291 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:02:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:58 smithi104 ceph-mon[32726]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-04-25T13:02:59.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:02:58 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3753781259' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:02:59.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:58 smithi178 ceph-mon[33233]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-04-25T13:02:59.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:02:58 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3753781259' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:02:59.543 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:01.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:01 smithi038 ceph-mon[29400]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2024-04-25T13:03:01.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:01 smithi104 ceph-mon[32726]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 -- 172.21.15.38:0/2376377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0102ac0 msgr2=0x7f26b0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 --2- 172.21.15.38:0/2376377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0102ac0 0x7f26b0102f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009990 tx=0x7f26a4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 -- 172.21.15.38:0/2376377687 shutdown_connections 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 --2- 172.21.15.38:0/2376377687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0108a70 0x7f26b0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 --2- 172.21.15.38:0/2376377687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26b0103480 0x7f26b010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 --2- 172.21.15.38:0/2376377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0102ac0 0x7f26b0102f40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 -- 172.21.15.38:0/2376377687 >> 172.21.15.38:0/2376377687 conn(0x7f26b0079bb0 msgr2=0x7f26b0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 -- 172.21.15.38:0/2376377687 shutdown_connections 2024-04-25T13:03:01.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.379+0000 7f26b58f9700 1 -- 172.21.15.38:0/2376377687 wait complete. 2024-04-25T13:03:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 Processor -- start 2024-04-25T13:03:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 -- start start 2024-04-25T13:03:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0102ac0 0x7f26b0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26b0103480 0x7f26b0199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 0x7f26b019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112dc0 con 0x7f26b0108a70 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112c40 con 0x7f26b0102ac0 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.380+0000 7f26b58f9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112f40 con 0x7f26b0103480 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 0x7f26b019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 0x7f26b019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50826/0 (socket says 172.21.15.38:50826) 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 -- 172.21.15.38:0/1053278348 learned_addr learned my addr 172.21.15.38:0/1053278348 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26ae7fc700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26b0103480 0x7f26b0199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26aeffd700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0102ac0 0x7f26b0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 -- 172.21.15.38:0/1053278348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26b0103480 msgr2=0x7f26b0199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26b0103480 0x7f26b0199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 -- 172.21.15.38:0/1053278348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0102ac0 msgr2=0x7f26b0199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0102ac0 0x7f26b0199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 -- 172.21.15.38:0/1053278348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b019e1d0 con 0x7f26b0108a70 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26aeffd700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0102ac0 0x7f26b0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:01.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26af7fe700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 0x7f26b019da90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26a800b970 tx=0x7f26a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8019070 con 0x7f26b0108a70 2024-04-25T13:03:01.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f26a800ce00 con 0x7f26b0108a70 2024-04-25T13:03:01.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8007370 con 0x7f26b0108a70 2024-04-25T13:03:01.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b019e4c0 con 0x7f26b0108a70 2024-04-25T13:03:01.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.381+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a5d40 con 0x7f26b0108a70 2024-04-25T13:03:01.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.383+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b0069730 con 0x7f26b0108a70 2024-04-25T13:03:01.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.384+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f26a801c070 con 0x7f26b0108a70 2024-04-25T13:03:01.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.384+0000 7f26b48f7700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26980a3cd0 0x7f26980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:01.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.384+0000 7f26aeffd700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26980a3cd0 0x7f26980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:01.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.384+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f26a80cf630 con 0x7f26b0108a70 2024-04-25T13:03:01.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.385+0000 7f26aeffd700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26980a3cd0 0x7f26980a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005fd0 tx=0x7f26a0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:01.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.389+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26a809c170 con 0x7f26b0108a70 2024-04-25T13:03:01.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:01 smithi178 ceph-mon[33233]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2024-04-25T13:03:01.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.597+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26b019a550 con 0x7f26b0108a70 2024-04-25T13:03:01.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.598+0000 7f26b48f7700 1 -- 172.21.15.38:0/1053278348 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22172 (secure 0 0 0) 0x7f26b019a550 con 0x7f26b0108a70 2024-04-25T13:03:01.598 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:03:01.598 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:02:45.729306+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.600+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26980a3cd0 msgr2=0x7f26980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.600+0000 7f26b58f9700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26980a3cd0 0x7f26980a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005fd0 tx=0x7f26a0005e30 comp rx=0 tx=0).stop 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.600+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 msgr2=0x7f26b019da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.600+0000 7f26b58f9700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 0x7f26b019da90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26a800b970 tx=0x7f26a800bc80 comp rx=0 tx=0).stop 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 shutdown_connections 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f26980a3cd0 0x7f26980a6190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26b0102ac0 0x7f26b0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26b0108a70 0x7f26b019da90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 --2- 172.21.15.38:0/1053278348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26b0103480 0x7f26b0199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 >> 172.21.15.38:0/1053278348 conn(0x7f26b0079bb0 msgr2=0x7f26b01016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 shutdown_connections 2024-04-25T13:03:01.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:01.601+0000 7f26b58f9700 1 -- 172.21.15.38:0/1053278348 wait complete. 2024-04-25T13:03:02.309 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:02 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1053278348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:02.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:02 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1053278348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:02.341 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:03:02.341 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:03:02.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:02 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1053278348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:03.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:03 smithi038 ceph-mon[29400]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2024-04-25T13:03:03.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:03 smithi104 ceph-mon[32726]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2024-04-25T13:03:03.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:03 smithi178 ceph-mon[33233]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 -- 172.21.15.38:0/2549269216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038103450 msgr2=0x7f503810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 --2- 172.21.15.38:0/2549269216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038103450 0x7f503810f8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f503000b0a0 tx=0x7f503000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 -- 172.21.15.38:0/2549269216 shutdown_connections 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 --2- 172.21.15.38:0/2549269216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038108a50 0x7f5038108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 --2- 172.21.15.38:0/2549269216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038103450 0x7f503810f8b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 --2- 172.21.15.38:0/2549269216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038102a90 0x7f5038102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 -- 172.21.15.38:0/2549269216 >> 172.21.15.38:0/2549269216 conn(0x7f5038079ba0 msgr2=0x7f5038079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 -- 172.21.15.38:0/2549269216 shutdown_connections 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 -- 172.21.15.38:0/2549269216 wait complete. 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 Processor -- start 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 -- start start 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.338+0000 7f503f4f3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038102a90 0x7f50381992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038103450 0x7f50381997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 0x7f503819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112da0 con 0x7f5038108a50 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112c20 con 0x7f5038102a90 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112f20 con 0x7f5038103450 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 0x7f503819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 0x7f503819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50848/0 (socket says 172.21.15.38:50848) 2024-04-25T13:03:04.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 -- 172.21.15.38:0/1683472183 learned_addr learned my addr 172.21.15.38:0/1683472183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503d28f700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038102a90 0x7f50381992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 -- 172.21.15.38:0/1683472183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038103450 msgr2=0x7f50381997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503ca8e700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038103450 0x7f50381997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038103450 0x7f50381997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 -- 172.21.15.38:0/1683472183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038102a90 msgr2=0x7f50381992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038102a90 0x7f50381992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 -- 172.21.15.38:0/1683472183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503819e2c0 con 0x7f5038108a50 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503d28f700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038102a90 0x7f50381992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503ca8e700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038103450 0x7f50381997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503da90700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 0x7f503819db80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f503000b070 tx=0x7f5030007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030021070 con 0x7f5038108a50 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503819e550 con 0x7f5038108a50 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f503001db20 con 0x7f5038108a50 2024-04-25T13:03:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503001bc50 con 0x7f5038108a50 2024-04-25T13:03:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.339+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a5e30 con 0x7f5038108a50 2024-04-25T13:03:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.340+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5038069730 con 0x7f5038108a50 2024-04-25T13:03:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.341+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5030024070 con 0x7f5038108a50 2024-04-25T13:03:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.342+0000 7f502e7fc700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f50240a3cd0 0x7f50240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.342+0000 7f503d28f700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f50240a3cd0 0x7f50240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.342+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f503000f020 con 0x7f5038108a50 2024-04-25T13:03:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.343+0000 7f503d28f700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f50240a3cd0 0x7f50240a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5034005fd0 tx=0x7f5034005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:04.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.347+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f503002b020 con 0x7f5038108a50 2024-04-25T13:03:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.557+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f503802df80 con 0x7f5038108a50 2024-04-25T13:03:04.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.559+0000 7f502e7fc700 1 -- 172.21.15.38:0/1683472183 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22172 (secure 0 0 0) 0x7f503009e520 con 0x7f5038108a50 2024-04-25T13:03:04.559 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:03:04.559 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:02:45.729306+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f50240a3cd0 msgr2=0x7f50240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f50240a3cd0 0x7f50240a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5034005fd0 tx=0x7f5034005ee0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 msgr2=0x7f503819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 0x7f503819db80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f503000b070 tx=0x7f5030007ad0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 shutdown_connections 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f50240a3cd0 0x7f50240a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5038102a90 0x7f50381992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5038108a50 0x7f503819db80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 --2- 172.21.15.38:0/1683472183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5038103450 0x7f50381997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 >> 172.21.15.38:0/1683472183 conn(0x7f5038079ba0 msgr2=0x7f5038101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 shutdown_connections 2024-04-25T13:03:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:04.561+0000 7f503f4f3700 1 -- 172.21.15.38:0/1683472183 wait complete. 2024-04-25T13:03:05.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:05 smithi038 ceph-mon[29400]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.0 KiB/s wr, 18 op/s 2024-04-25T13:03:05.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:05 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1683472183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:05.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:03:05.272 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:03:05.272 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:03:05.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:03:05.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:03:05.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:03:05.274 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:03:05.274 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:03:05.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:03:05.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:03:05.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:03:05.276 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:03:05.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:05 smithi104 ceph-mon[32726]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.0 KiB/s wr, 18 op/s 2024-04-25T13:03:05.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:05 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1683472183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:05.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:05 smithi178 ceph-mon[33233]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.0 KiB/s wr, 18 op/s 2024-04-25T13:03:05.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:05 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1683472183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:03:06.033 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.034 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.034 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.064 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.064 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.083 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.083 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.086 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:06.149 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:07.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:07 smithi038 ceph-mon[29400]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2024-04-25T13:03:07.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:07 smithi104 ceph-mon[32726]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2024-04-25T13:03:07.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:07 smithi178 ceph-mon[33233]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2024-04-25T13:03:09.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:09 smithi038 ceph-mon[29400]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-25T13:03:09.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:09 smithi104 ceph-mon[32726]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-25T13:03:09.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:09 smithi178 ceph-mon[33233]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-25T13:03:11.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:11 smithi038 ceph-mon[29400]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-25T13:03:11.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:11 smithi104 ceph-mon[32726]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-25T13:03:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:11 smithi178 ceph-mon[33233]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-25T13:03:13.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:13 smithi038 ceph-mon[29400]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-25T13:03:13.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:13 smithi104 ceph-mon[32726]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-25T13:03:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:13 smithi178 ceph-mon[33233]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-25T13:03:15.286 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:15 smithi038 ceph-mon[29400]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T13:03:15.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:15 smithi104 ceph-mon[32726]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T13:03:15.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:15 smithi178 ceph-mon[33233]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T13:03:17.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:17 smithi038 ceph-mon[29400]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T13:03:17.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:17 smithi104 ceph-mon[32726]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T13:03:17.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:17 smithi178 ceph-mon[33233]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T13:03:19.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:19 smithi038 ceph-mon[29400]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:03:19.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:19 smithi104 ceph-mon[32726]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:03:19.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 -- 172.21.15.38:0/617798009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84101940 msgr2=0x7f6f84101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:19.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/617798009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84101940 0x7f6f84101dc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009990 tx=0x7f6f78009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 -- 172.21.15.38:0/617798009 shutdown_connections 2024-04-25T13:03:19.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/617798009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f84107990 0x7f6f84107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/617798009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f84102300 0x7f6f8410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/617798009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84101940 0x7f6f84101dc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 -- 172.21.15.38:0/617798009 >> 172.21.15.38:0/617798009 conn(0x7f6f840788f0 msgr2=0x7f6f84078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 -- 172.21.15.38:0/617798009 shutdown_connections 2024-04-25T13:03:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 -- 172.21.15.38:0/617798009 wait complete. 2024-04-25T13:03:19.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 Processor -- start 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.422+0000 7f6f8a47b700 1 -- start start 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f8a47b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f84101940 0x7f6f84199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f8a47b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f84102300 0x7f6f841996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f8a47b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84107990 0x7f6f8419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f8a47b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84111c70 con 0x7f6f84107990 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f8a47b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84111af0 con 0x7f6f84102300 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f8a47b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84111df0 con 0x7f6f84101940 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84107990 0x7f6f8419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84107990 0x7f6f8419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48486/0 (socket says 172.21.15.38:48486) 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 -- 172.21.15.38:0/4050776332 learned_addr learned my addr 172.21.15.38:0/4050776332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f83fff700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f84101940 0x7f6f84199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f837fe700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f84102300 0x7f6f841996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:19.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 -- 172.21.15.38:0/4050776332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f84101940 msgr2=0x7f6f84199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f84101940 0x7f6f84199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 -- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f84102300 msgr2=0x7f6f841996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f84102300 0x7f6f841996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 -- 172.21.15.38:0/4050776332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8419e1a0 con 0x7f6f84107990 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.423+0000 7f6f88a18700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84107990 0x7f6f8419da60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00ada0 tx=0x7f6f7c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.424+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c017020 con 0x7f6f84107990 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.424+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6f7c00cda0 con 0x7f6f84107990 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.424+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8419e490 con 0x7f6f84107990 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.424+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c005310 con 0x7f6f84107990 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.424+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f841a5d10 con 0x7f6f84107990 2024-04-25T13:03:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.424+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f84063190 con 0x7f6f84107990 2024-04-25T13:03:19.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.426+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f7c00c8d0 con 0x7f6f84107990 2024-04-25T13:03:19.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.426+0000 7f6f817fa700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6f6c0a3cd0 0x7f6f6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:19.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.426+0000 7f6f83fff700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6f6c0a3cd0 0x7f6f6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:19.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.426+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6f7c0d0530 con 0x7f6f84107990 2024-04-25T13:03:19.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.427+0000 7f6f817fa700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] conn(0x7f6f6c0ad4b0 0x7f6f6c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.427+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 --> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f7c023e50 con 0x7f6f6c0ad4b0 2024-04-25T13:03:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.427+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6f7c0d0a10 con 0x7f6f84107990 2024-04-25T13:03:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.427+0000 7f6f83fff700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6f6c0a3cd0 0x7f6f6c0a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005950 tx=0x7f6f7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.427+0000 7f6f837fe700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] conn(0x7f6f6c0ad4b0 0x7f6f6c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.428+0000 7f6f837fe700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] conn(0x7f6f6c0ad4b0 0x7f6f6c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.428+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== osd.4 v2:172.21.15.104:6808/309827173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f7c023e50 con 0x7f6f6c0ad4b0 2024-04-25T13:03:19.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 --> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f84063610 con 0x7f6f6c0ad4b0 2024-04-25T13:03:19.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f817fa700 1 -- 172.21.15.38:0/4050776332 <== osd.4 v2:172.21.15.104:6808/309827173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f84063610 con 0x7f6f6c0ad4b0 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] conn(0x7f6f6c0ad4b0 msgr2=0x7f6f6c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] conn(0x7f6f6c0ad4b0 0x7f6f6c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6f6c0a3cd0 msgr2=0x7f6f6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f6f6c0a3cd0 0x7f6f6c0a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005950 tx=0x7f6f7400b410 comp rx=0 tx=0).stop 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84107990 msgr2=0x7f6f8419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.445+0000 7f6f8a47b700 1 --2- 172.21.15.38:0/4050776332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84107990 0x7f6f8419da60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00ada0 tx=0x7f6f7c00c330 comp rx=0 tx=0).stop 2024-04-25T13:03:19.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.446+0000 7f6f88a18700 1 -- 172.21.15.38:0/4050776332 reap_dead start 2024-04-25T13:03:19.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.446+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 shutdown_connections 2024-04-25T13:03:19.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.446+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 >> 172.21.15.38:0/4050776332 conn(0x7f6f840788f0 msgr2=0x7f6f8406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:19.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.446+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 shutdown_connections 2024-04-25T13:03:19.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:19.446+0000 7f6f8a47b700 1 -- 172.21.15.38:0/4050776332 wait complete. 2024-04-25T13:03:19.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:19 smithi178 ceph-mon[33233]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:03:20.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 -- 172.21.15.38:0/2033224809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230101030 msgr2=0x7f0230101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:20.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 --2- 172.21.15.38:0/2033224809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230101030 0x7f0230101490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0220009990 tx=0x7f0220009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 -- 172.21.15.38:0/2033224809 shutdown_connections 2024-04-25T13:03:20.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 --2- 172.21.15.38:0/2033224809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0230108ad0 0x7f0230108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 --2- 172.21.15.38:0/2033224809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02301019d0 0x7f023010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 --2- 172.21.15.38:0/2033224809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230101030 0x7f0230101490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 -- 172.21.15.38:0/2033224809 >> 172.21.15.38:0/2033224809 conn(0x7f0230077fb0 msgr2=0x7f02300ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 -- 172.21.15.38:0/2033224809 shutdown_connections 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.772+0000 7f02380e9700 1 -- 172.21.15.38:0/2033224809 wait complete. 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 Processor -- start 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 -- start start 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0230101030 0x7f0230102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02301019d0 0x7f0230102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230108ad0 0x7f0230102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0230112d60 con 0x7f0230108ad0 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0230112be0 con 0x7f02301019d0 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f02380e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0230112ee0 con 0x7f0230101030 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f0236686700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230108ad0 0x7f0230102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f0236686700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230108ad0 0x7f0230102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53882/0 (socket says 172.21.15.38:53882) 2024-04-25T13:03:20.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f0236686700 1 -- 172.21.15.38:0/1358420696 learned_addr learned my addr 172.21.15.38:0/1358420696 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.773+0000 7f0235684700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02301019d0 0x7f0230102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0235e85700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0230101030 0x7f0230102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0236686700 1 -- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0230101030 msgr2=0x7f0230102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0236686700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0230101030 0x7f0230102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0236686700 1 -- 172.21.15.38:0/1358420696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02301019d0 msgr2=0x7f0230102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0236686700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02301019d0 0x7f0230102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0236686700 1 -- 172.21.15.38:0/1358420696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301a7700 con 0x7f0230108ad0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0236686700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230108ad0 0x7f0230102c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0228007ee0 tx=0x7f022800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228004bb0 con 0x7f0230108ad0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301a79f0 con 0x7f0230108ad0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0228004d50 con 0x7f0230108ad0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a7f50 con 0x7f0230108ad0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.774+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280059e0 con 0x7f0230108ad0 2024-04-25T13:03:20.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.775+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02300630d0 con 0x7f0230108ad0 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.777+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f022801a020 con 0x7f0230108ad0 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.777+0000 7f0226ffd700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f021c0a3d50 0x7f021c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0235e85700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f021c0a3d50 0x7f021c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f02280d0540 con 0x7f0230108ad0 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0226ffd700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] conn(0x7f021c0ad530 0x7f021c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 --> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f022801de50 con 0x7f021c0ad530 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f02280d08f0 con 0x7f0230108ad0 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0235e85700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f021c0a3d50 0x7f021c0a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f022c005950 tx=0x7f022c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.778+0000 7f0235684700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] conn(0x7f021c0ad530 0x7f021c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.779+0000 7f0235684700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] conn(0x7f021c0ad530 0x7f021c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.780+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== osd.11 v2:172.21.15.178:6824/700919223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f022801de50 con 0x7f021c0ad530 2024-04-25T13:03:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.796+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 --> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0230063550 con 0x7f021c0ad530 2024-04-25T13:03:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f0226ffd700 1 -- 172.21.15.38:0/1358420696 <== osd.11 v2:172.21.15.178:6824/700919223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0230063550 con 0x7f021c0ad530 2024-04-25T13:03:20.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] conn(0x7f021c0ad530 msgr2=0x7f021c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] conn(0x7f021c0ad530 0x7f021c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f021c0a3d50 msgr2=0x7f021c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f021c0a3d50 0x7f021c0a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f022c005950 tx=0x7f022c00a300 comp rx=0 tx=0).stop 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230108ad0 msgr2=0x7f0230102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 --2- 172.21.15.38:0/1358420696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0230108ad0 0x7f0230102c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0228007ee0 tx=0x7f022800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f0236686700 1 -- 172.21.15.38:0/1358420696 reap_dead start 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 shutdown_connections 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.797+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 >> 172.21.15.38:0/1358420696 conn(0x7f0230077fb0 msgr2=0x7f02300ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.798+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 shutdown_connections 2024-04-25T13:03:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:20.798+0000 7f02380e9700 1 -- 172.21.15.38:0/1358420696 wait complete. 2024-04-25T13:03:21.320 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:21 smithi038 ceph-mon[29400]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:03:21.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:21 smithi104 ceph-mon[32726]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 -- 172.21.15.38:0/1658985453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4101940 msgr2=0x7ffaf4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/1658985453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4101940 0x7ffaf4101dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009990 tx=0x7ffae8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 -- 172.21.15.38:0/1658985453 shutdown_connections 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/1658985453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf4107990 0x7ffaf4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/1658985453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaf4102300 0x7ffaf410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/1658985453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4101940 0x7ffaf4101dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.325+0000 7ffaf88c8700 1 -- 172.21.15.38:0/1658985453 >> 172.21.15.38:0/1658985453 conn(0x7ffaf40788f0 msgr2=0x7ffaf4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.326+0000 7ffaf88c8700 1 -- 172.21.15.38:0/1658985453 shutdown_connections 2024-04-25T13:03:21.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.326+0000 7ffaf88c8700 1 -- 172.21.15.38:0/1658985453 wait complete. 2024-04-25T13:03:21.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.326+0000 7ffaf88c8700 1 Processor -- start 2024-04-25T13:03:21.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 -- start start 2024-04-25T13:03:21.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaf4101940 0x7ffaf41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf4102300 0x7ffaf4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4107990 0x7ffaf419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4111c70 con 0x7ffaf4107990 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4111af0 con 0x7ffaf4102300 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf88c8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4111df0 con 0x7ffaf4101940 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4107990 0x7ffaf419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4107990 0x7ffaf419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53902/0 (socket says 172.21.15.38:53902) 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 -- 172.21.15.38:0/3144178459 learned_addr learned my addr 172.21.15.38:0/3144178459 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf259c700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaf4101940 0x7ffaf41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf1d9b700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf4102300 0x7ffaf4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 -- 172.21.15.38:0/3144178459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaf4101940 msgr2=0x7ffaf41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaf4101940 0x7ffaf41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 -- 172.21.15.38:0/3144178459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf4102300 msgr2=0x7ffaf4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf4102300 0x7ffaf4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 -- 172.21.15.38:0/3144178459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf419d0f0 con 0x7ffaf4107990 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf1d9b700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf4102300 0x7ffaf4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.327+0000 7ffaf2d9d700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4107990 0x7ffaf419c9b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffaec007d70 tx=0x7ffaec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.328+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec016020 con 0x7ffaf4107990 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.328+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf419d3e0 con 0x7ffaf4107990 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.328+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ffaec00ad40 con 0x7ffaf4107990 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.328+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec011620 con 0x7ffaf4107990 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.328+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf41a4c60 con 0x7ffaf4107990 2024-04-25T13:03:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.328+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaf4063150 con 0x7ffaf4107990 2024-04-25T13:03:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.330+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffaec019020 con 0x7ffaf4107990 2024-04-25T13:03:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.330+0000 7ffae37fe700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffadc0a3cd0 0x7ffadc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.330+0000 7ffaf259c700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffadc0a3cd0 0x7ffadc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.330+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffaec0d08c0 con 0x7ffaf4107990 2024-04-25T13:03:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.331+0000 7ffae37fe700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] conn(0x7ffadc0ad4b0 0x7ffadc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.331+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 --> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffadc0aff80 con 0x7ffadc0ad4b0 2024-04-25T13:03:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.331+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ffaec0d0da0 con 0x7ffaf4107990 2024-04-25T13:03:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.331+0000 7ffaf1d9b700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] conn(0x7ffadc0ad4b0 0x7ffadc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.331+0000 7ffaf259c700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffadc0a3cd0 0x7ffadc0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffae4005950 tx=0x7ffae400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.331+0000 7ffaf1d9b700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] conn(0x7ffadc0ad4b0 0x7ffadc0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.332+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== osd.3 v2:172.21.15.38:6810/1943053505 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffadc0aff80 con 0x7ffadc0ad4b0 2024-04-25T13:03:21.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.349+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 --> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaf40635d0 con 0x7ffadc0ad4b0 2024-04-25T13:03:21.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffae37fe700 1 -- 172.21.15.38:0/3144178459 <== osd.3 v2:172.21.15.38:6810/1943053505 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffaf40635d0 con 0x7ffadc0ad4b0 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] conn(0x7ffadc0ad4b0 msgr2=0x7ffadc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] conn(0x7ffadc0ad4b0 0x7ffadc0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffadc0a3cd0 msgr2=0x7ffadc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ffadc0a3cd0 0x7ffadc0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffae4005950 tx=0x7ffae400b410 comp rx=0 tx=0).stop 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4107990 msgr2=0x7ffaf419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 --2- 172.21.15.38:0/3144178459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaf4107990 0x7ffaf419c9b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffaec007d70 tx=0x7ffaec00a340 comp rx=0 tx=0).stop 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf2d9d700 1 -- 172.21.15.38:0/3144178459 reap_dead start 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 shutdown_connections 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 >> 172.21.15.38:0/3144178459 conn(0x7ffaf40788f0 msgr2=0x7ffaf41006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 shutdown_connections 2024-04-25T13:03:21.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.350+0000 7ffaf88c8700 1 -- 172.21.15.38:0/3144178459 wait complete. 2024-04-25T13:03:21.488 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:21 smithi178 ceph-mon[33233]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 -- 172.21.15.38:0/1586752607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0101940 msgr2=0x7f3af0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 --2- 172.21.15.38:0/1586752607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0101940 0x7f3af0101dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009990 tx=0x7f3ae4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 -- 172.21.15.38:0/1586752607 shutdown_connections 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 --2- 172.21.15.38:0/1586752607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3af0102300 0x7f3af010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 --2- 172.21.15.38:0/1586752607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3af0107990 0x7f3af0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 --2- 172.21.15.38:0/1586752607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0101940 0x7f3af0101dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 -- 172.21.15.38:0/1586752607 >> 172.21.15.38:0/1586752607 conn(0x7f3af00788f0 msgr2=0x7f3af0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 -- 172.21.15.38:0/1586752607 shutdown_connections 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.543+0000 7f3af52b3700 1 -- 172.21.15.38:0/1586752607 wait complete. 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 Processor -- start 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 -- start start 2024-04-25T13:03:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3af0101940 0x7f3af0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3af0102300 0x7f3af0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0107990 0x7f3af019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0111c70 con 0x7f3af0107990 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0111af0 con 0x7f3af0101940 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3af52b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0111df0 con 0x7f3af0102300 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.544+0000 7f3aef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0107990 0x7f3af019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0107990 0x7f3af019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53918/0 (socket says 172.21.15.38:53918) 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 -- 172.21.15.38:0/692374518 learned_addr learned my addr 172.21.15.38:0/692374518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aeeffd700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3af0101940 0x7f3af0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aee7fc700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3af0102300 0x7f3af0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 -- 172.21.15.38:0/692374518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3af0102300 msgr2=0x7f3af0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3af0102300 0x7f3af0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 -- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3af0101940 msgr2=0x7f3af0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3af0101940 0x7f3af0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 -- 172.21.15.38:0/692374518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af019d0c0 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3aef7fe700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0107990 0x7f3af019c980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800ef30 tx=0x7f3ae800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80092e0 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af019d350 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3ae8004bb0 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8004280 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.545+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af01a4c30 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.546+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3af00631d0 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.547+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3ae801b030 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3ad7fff700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3ad80a3cd0 0x7f3ad80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3aeeffd700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3ad80a3cd0 0x7f3ad80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3ae80d08e0 con 0x7f3af0107990 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3ad7fff700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] conn(0x7f3ad80ad4b0 0x7f3ad80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 --> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ad80aff80 con 0x7f3ad80ad4b0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3ae80d0dc0 con 0x7f3af0107990 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.548+0000 7f3aeeffd700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3ad80a3cd0 0x7f3ad80a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009be0 tx=0x7f3ae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.549+0000 7f3aee7fc700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] conn(0x7f3ad80ad4b0 0x7f3ad80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.549+0000 7f3aee7fc700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] conn(0x7f3ad80ad4b0 0x7f3ad80af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.550+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== osd.10 v2:172.21.15.104:6824/2081501725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ad80aff80 con 0x7f3ad80ad4b0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.566+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 --> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3af0063650 con 0x7f3ad80ad4b0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3ad7fff700 1 -- 172.21.15.38:0/692374518 <== osd.10 v2:172.21.15.104:6824/2081501725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3af0063650 con 0x7f3ad80ad4b0 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] conn(0x7f3ad80ad4b0 msgr2=0x7f3ad80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] conn(0x7f3ad80ad4b0 0x7f3ad80af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3ad80a3cd0 msgr2=0x7f3ad80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3ad80a3cd0 0x7f3ad80a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009be0 tx=0x7f3ae0009260 comp rx=0 tx=0).stop 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0107990 msgr2=0x7f3af019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 --2- 172.21.15.38:0/692374518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af0107990 0x7f3af019c980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800ef30 tx=0x7f3ae800c3a0 comp rx=0 tx=0).stop 2024-04-25T13:03:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3aef7fe700 1 -- 172.21.15.38:0/692374518 reap_dead start 2024-04-25T13:03:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 shutdown_connections 2024-04-25T13:03:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 >> 172.21.15.38:0/692374518 conn(0x7f3af00788f0 msgr2=0x7f3af01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 shutdown_connections 2024-04-25T13:03:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:21.567+0000 7f3af52b3700 1 -- 172.21.15.38:0/692374518 wait complete. 2024-04-25T13:03:22.812 INFO:teuthology.orchestra.run.smithi038.stdout:313532612633 2024-04-25T13:03:22.813 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:03:23.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 -- 172.21.15.38:0/1274743886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4103450 msgr2=0x7fdde410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:23.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/1274743886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4103450 0x7fdde410f8b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00b0a0 tx=0x7fdddc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 -- 172.21.15.38:0/1274743886 shutdown_connections 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/1274743886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4103450 0x7fdde410f8b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/1274743886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde4108a50 0x7fdde4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/1274743886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde4102a90 0x7fdde4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 -- 172.21.15.38:0/1274743886 >> 172.21.15.38:0/1274743886 conn(0x7fdde4079ba0 msgr2=0x7fdde4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 -- 172.21.15.38:0/1274743886 shutdown_connections 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.012+0000 7fdde9e5d700 1 -- 172.21.15.38:0/1274743886 wait complete. 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 Processor -- start 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 -- start start 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4102a90 0x7fdde41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde4103450 0x7fdde41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:23.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde4108a50 0x7fdde419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112da0 con 0x7fdde4102a90 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112c20 con 0x7fdde4103450 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde9e5d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112f20 con 0x7fdde4108a50 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4102a90 0x7fdde41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4102a90 0x7fdde41991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53946/0 (socket says 172.21.15.38:53946) 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde37fe700 1 -- 172.21.15.38:0/4169497102 learned_addr learned my addr 172.21.15.38:0/4169497102 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde2ffd700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde4103450 0x7fdde41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde3fff700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde4108a50 0x7fdde419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.013+0000 7fdde37fe700 1 -- 172.21.15.38:0/4169497102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde4108a50 msgr2=0x7fdde419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde37fe700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde4108a50 0x7fdde419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde37fe700 1 -- 172.21.15.38:0/4169497102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde4103450 msgr2=0x7fdde41996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde37fe700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde4103450 0x7fdde41996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde37fe700 1 -- 172.21.15.38:0/4169497102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde419e1c0 con 0x7fdde4102a90 2024-04-25T13:03:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde2ffd700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde4103450 0x7fdde41996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde37fe700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4102a90 0x7fdde41991b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fddd400b530 tx=0x7fddd400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4015020 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde419e4b0 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fddd4011d00 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4010e60 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.014+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a5d30 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.015+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdde40631d0 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.016+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fddd4018020 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde0ff9700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fddcc0a3cd0 0x7fddcc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde2ffd700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fddcc0a3cd0 0x7fddcc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fddd40d0250 con 0x7fdde4102a90 2024-04-25T13:03:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde0ff9700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] conn(0x7fddcc0ad4b0 0x7fddcc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 --> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddcc0aff80 con 0x7fddcc0ad4b0 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fddd40d0730 con 0x7fdde4102a90 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde3fff700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] conn(0x7fddcc0ad4b0 0x7fddcc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde2ffd700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fddcc0a3cd0 0x7fddcc0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdde407a1a0 tx=0x7fddd800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.017+0000 7fdde3fff700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] conn(0x7fddcc0ad4b0 0x7fddcc0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.018+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== osd.9 v2:172.21.15.38:6826/1397570004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddcc0aff80 con 0x7fddcc0ad4b0 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 --> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdde4063650 con 0x7fddcc0ad4b0 2024-04-25T13:03:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde0ff9700 1 -- 172.21.15.38:0/4169497102 <== osd.9 v2:172.21.15.38:6826/1397570004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdde4063650 con 0x7fddcc0ad4b0 2024-04-25T13:03:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] conn(0x7fddcc0ad4b0 msgr2=0x7fddcc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] conn(0x7fddcc0ad4b0 0x7fddcc0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fddcc0a3cd0 msgr2=0x7fddcc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fddcc0a3cd0 0x7fddcc0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdde407a1a0 tx=0x7fddd800b5f0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4102a90 msgr2=0x7fdde41991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 --2- 172.21.15.38:0/4169497102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4102a90 0x7fdde41991b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fddd400b530 tx=0x7fddd400b8f0 comp rx=0 tx=0).stop 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde3fff700 1 -- 172.21.15.38:0/4169497102 reap_dead start 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 shutdown_connections 2024-04-25T13:03:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.035+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 >> 172.21.15.38:0/4169497102 conn(0x7fdde4079ba0 msgr2=0x7fdde4101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.036+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 shutdown_connections 2024-04-25T13:03:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:23.036+0000 7fdde9e5d700 1 -- 172.21.15.38:0/4169497102 wait complete. 2024-04-25T13:03:23.289 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:23 smithi038 ceph-mon[29400]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:03:23.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:23 smithi104 ceph-mon[32726]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:03:23.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:23 smithi178 ceph-mon[33233]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 -- 172.21.15.38:0/1181058416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4107960 msgr2=0x7fd5e4107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 --2- 172.21.15.38:0/1181058416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4107960 0x7fd5e4107d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009990 tx=0x7fd5e0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 -- 172.21.15.38:0/1181058416 shutdown_connections 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 --2- 172.21.15.38:0/1181058416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5e4102320 0x7fd5e410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 --2- 172.21.15.38:0/1181058416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4107960 0x7fd5e4107d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 --2- 172.21.15.38:0/1181058416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5e4101960 0x7fd5e4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 -- 172.21.15.38:0/1181058416 >> 172.21.15.38:0/1181058416 conn(0x7fd5e40789e0 msgr2=0x7fd5e4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 -- 172.21.15.38:0/1181058416 shutdown_connections 2024-04-25T13:03:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.462+0000 7fd5eb945700 1 -- 172.21.15.38:0/1181058416 wait complete. 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.463+0000 7fd5eb945700 1 Processor -- start 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.463+0000 7fd5eb945700 1 -- start start 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.463+0000 7fd5eb945700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4101960 0x7fd5e4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5e4102320 0x7fd5e4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5e4107960 0x7fd5e419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4111d40 con 0x7fd5e4101960 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4111bc0 con 0x7fd5e4107960 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4111ec0 con 0x7fd5e4102320 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4101960 0x7fd5e4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4101960 0x7fd5e4199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53966/0 (socket says 172.21.15.38:53966) 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 -- 172.21.15.38:0/2999567297 learned_addr learned my addr 172.21.15.38:0/2999567297 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e8ee0700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5e4102320 0x7fd5e4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 -- 172.21.15.38:0/2999567297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5e4102320 msgr2=0x7fd5e4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5e4102320 0x7fd5e4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 -- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5e4107960 msgr2=0x7fd5e419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5e4107960 0x7fd5e419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 -- 172.21.15.38:0/2999567297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e419e310 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e8ee0700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5e4102320 0x7fd5e4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5e96e1700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4101960 0x7fd5e4199300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c00 tx=0x7fd5e00037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0027070 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd5e00041e0 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e419e5a0 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000f460 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.464+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a5e80 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.465+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5e406b480 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.466+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd5e002a070 con 0x7fd5e4101960 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5da7fc700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd5d00a3d50 0x7fd5d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5e8ee0700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd5d00a3d50 0x7fd5d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd5e00d1650 con 0x7fd5e4101960 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5da7fc700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] conn(0x7fd5d00ad530 0x7fd5d00af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 --> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5e0019e50 con 0x7fd5d00ad530 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd5e00d1b30 con 0x7fd5e4101960 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5e8ee0700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd5d00a3d50 0x7fd5d00a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5e406c250 tx=0x7fd5d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.467+0000 7fd5e9ee2700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] conn(0x7fd5d00ad530 0x7fd5d00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.468+0000 7fd5e9ee2700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] conn(0x7fd5d00ad530 0x7fd5d00af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:24.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.469+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== osd.1 v2:172.21.15.104:6800/739328309 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5e0019e50 con 0x7fd5d00ad530 2024-04-25T13:03:24.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.485+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 --> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5e41030a0 con 0x7fd5d00ad530 2024-04-25T13:03:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5da7fc700 1 -- 172.21.15.38:0/2999567297 <== osd.1 v2:172.21.15.104:6800/739328309 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5e41030a0 con 0x7fd5d00ad530 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] conn(0x7fd5d00ad530 msgr2=0x7fd5d00af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] conn(0x7fd5d00ad530 0x7fd5d00af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd5d00a3d50 msgr2=0x7fd5d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd5d00a3d50 0x7fd5d00a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5e406c250 tx=0x7fd5d4008040 comp rx=0 tx=0).stop 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4101960 msgr2=0x7fd5e4199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 --2- 172.21.15.38:0/2999567297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5e4101960 0x7fd5e4199300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c00 tx=0x7fd5e00037a0 comp rx=0 tx=0).stop 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5e9ee2700 1 -- 172.21.15.38:0/2999567297 reap_dead start 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 shutdown_connections 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 >> 172.21.15.38:0/2999567297 conn(0x7fd5e40789e0 msgr2=0x7fd5e4100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 shutdown_connections 2024-04-25T13:03:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:24.486+0000 7fd5eb945700 1 -- 172.21.15.38:0/2999567297 wait complete. 2024-04-25T13:03:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:347892350995 2024-04-25T13:03:25.010 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:03:25.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:25 smithi038 ceph-mon[29400]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:03:25.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 -- 172.21.15.38:0/1459316637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc107990 msgr2=0x7f79bc107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 --2- 172.21.15.38:0/1459316637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc107990 0x7f79bc107d70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00b0a0 tx=0x7f79ac00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 -- 172.21.15.38:0/1459316637 shutdown_connections 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 --2- 172.21.15.38:0/1459316637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79bc102300 0x7f79bc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 --2- 172.21.15.38:0/1459316637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc107990 0x7f79bc107d70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 --2- 172.21.15.38:0/1459316637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79bc101940 0x7f79bc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 -- 172.21.15.38:0/1459316637 >> 172.21.15.38:0/1459316637 conn(0x7f79bc0788f0 msgr2=0x7f79bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 -- 172.21.15.38:0/1459316637 shutdown_connections 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.275+0000 7f79c21cf700 1 -- 172.21.15.38:0/1459316637 wait complete. 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 Processor -- start 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 -- start start 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79bc101940 0x7f79bc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc102300 0x7f79bc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79bc107990 0x7f79bc1a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc111c40 con 0x7f79bc102300 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc111ac0 con 0x7f79bc101940 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79c21cf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc111dc0 con 0x7f79bc107990 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79baffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc102300 0x7f79bc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79baffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc102300 0x7f79bc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53978/0 (socket says 172.21.15.38:53978) 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.276+0000 7f79baffd700 1 -- 172.21.15.38:0/2900987385 learned_addr learned my addr 172.21.15.38:0/2900987385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79bb7fe700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79bc101940 0x7f79bc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79baffd700 1 -- 172.21.15.38:0/2900987385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79bc107990 msgr2=0x7f79bc1a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79bbfff700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79bc107990 0x7f79bc1a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79baffd700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79bc107990 0x7f79bc1a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79baffd700 1 -- 172.21.15.38:0/2900987385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79bc101940 msgr2=0x7f79bc19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79baffd700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79bc101940 0x7f79bc19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79baffd700 1 -- 172.21.15.38:0/2900987385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc1a5640 con 0x7f79bc102300 2024-04-25T13:03:25.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79baffd700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc102300 0x7f79bc19a5c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d980 tx=0x7f79a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:25.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4016070 con 0x7f79bc102300 2024-04-25T13:03:25.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f79a4004d00 con 0x7f79bc102300 2024-04-25T13:03:25.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4005610 con 0x7f79bc102300 2024-04-25T13:03:25.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc1a5810 con 0x7f79bc102300 2024-04-25T13:03:25.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.277+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc1a5d20 con 0x7f79bc102300 2024-04-25T13:03:25.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.278+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79bc0630d0 con 0x7f79bc102300 2024-04-25T13:03:25.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f79a4004750 con 0x7f79bc102300 2024-04-25T13:03:25.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79b8ff9700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f79a80a3d50 0x7f79a80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f79a40d07d0 con 0x7f79bc102300 2024-04-25T13:03:25.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79bb7fe700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f79a80a3d50 0x7f79a80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79b8ff9700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] conn(0x7f79a80ad530 0x7f79a80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 --> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79a4015ec0 con 0x7f79a80ad530 2024-04-25T13:03:25.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.280+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f79a40d0cb0 con 0x7f79bc102300 2024-04-25T13:03:25.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.281+0000 7f79bbfff700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] conn(0x7f79a80ad530 0x7f79a80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.281+0000 7f79bb7fe700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f79a80a3d50 0x7f79a80a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009fd0 tx=0x7f79ac00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:25.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.281+0000 7f79bbfff700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] conn(0x7f79a80ad530 0x7f79a80af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:25.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.282+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== osd.0 v2:172.21.15.38:6802/4061500171 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79a4015ec0 con 0x7f79a80ad530 2024-04-25T13:03:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.298+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 --> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79bc063550 con 0x7f79a80ad530 2024-04-25T13:03:25.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79b8ff9700 1 -- 172.21.15.38:0/2900987385 <== osd.0 v2:172.21.15.38:6802/4061500171 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f79bc063550 con 0x7f79a80ad530 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] conn(0x7f79a80ad530 msgr2=0x7f79a80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] conn(0x7f79a80ad530 0x7f79a80af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f79a80a3d50 msgr2=0x7f79a80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f79a80a3d50 0x7f79a80a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009fd0 tx=0x7f79ac00b7b0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc102300 msgr2=0x7f79bc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 --2- 172.21.15.38:0/2900987385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79bc102300 0x7f79bc19a5c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d980 tx=0x7f79a400dc90 comp rx=0 tx=0).stop 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79bbfff700 1 -- 172.21.15.38:0/2900987385 reap_dead start 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 shutdown_connections 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 >> 172.21.15.38:0/2900987385 conn(0x7f79bc0788f0 msgr2=0x7f79bc1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 shutdown_connections 2024-04-25T13:03:25.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.299+0000 7f79c21cf700 1 -- 172.21.15.38:0/2900987385 wait complete. 2024-04-25T13:03:25.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:25 smithi104 ceph-mon[32726]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:03:25.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:25 smithi178 ceph-mon[33233]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 -- 172.21.15.38:0/425564996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0103450 msgr2=0x7fe8f010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/425564996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0103450 0x7fe8f010f8b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800b0a0 tx=0x7fe8e800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 -- 172.21.15.38:0/425564996 shutdown_connections 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/425564996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0103450 0x7fe8f010f8b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/425564996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f0108a50 0x7fe8f0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/425564996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f0102a90 0x7fe8f0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 -- 172.21.15.38:0/425564996 >> 172.21.15.38:0/425564996 conn(0x7fe8f0079ba0 msgr2=0x7fe8f0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 -- 172.21.15.38:0/425564996 shutdown_connections 2024-04-25T13:03:25.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.498+0000 7fe8f5d06700 1 -- 172.21.15.38:0/425564996 wait complete. 2024-04-25T13:03:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 Processor -- start 2024-04-25T13:03:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 -- start start 2024-04-25T13:03:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0102a90 0x7fe8f0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f0103450 0x7fe8f01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f0108a50 0x7fe8f019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0112da0 con 0x7fe8f0102a90 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0112c20 con 0x7fe8f0108a50 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.499+0000 7fe8f5d06700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0112f20 con 0x7fe8f0103450 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0102a90 0x7fe8f0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0102a90 0x7fe8f0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53988/0 (socket says 172.21.15.38:53988) 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 -- 172.21.15.38:0/1135740071 learned_addr learned my addr 172.21.15.38:0/1135740071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8effff700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f0108a50 0x7fe8f019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8e6ffd700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f0103450 0x7fe8f01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 -- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f0103450 msgr2=0x7fe8f01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f0103450 0x7fe8f01996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 -- 172.21.15.38:0/1135740071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f0108a50 msgr2=0x7fe8f019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f0108a50 0x7fe8f019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 -- 172.21.15.38:0/1135740071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019e1a0 con 0x7fe8f0102a90 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ef7fe700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0102a90 0x7fe8f0199190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000d980 tx=0x7fe8e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:25.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0017070 con 0x7fe8f0102a90 2024-04-25T13:03:25.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe8e0005020 con 0x7fe8f0102a90 2024-04-25T13:03:25.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0009bf0 con 0x7fe8f0102a90 2024-04-25T13:03:25.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f019e490 con 0x7fe8f0102a90 2024-04-25T13:03:25.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.500+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a5d10 con 0x7fe8f0102a90 2024-04-25T13:03:25.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.501+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8f0063150 con 0x7fe8f0102a90 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.502+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe8e00051d0 con 0x7fe8f0102a90 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.503+0000 7fe8ed7fa700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe8dc0a3cd0 0x7fe8dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.503+0000 7fe8e6ffd700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe8dc0a3cd0 0x7fe8dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.503+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe8e00d0790 con 0x7fe8f0102a90 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.503+0000 7fe8ed7fa700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] conn(0x7fe8dc0ad4b0 0x7fe8dc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.503+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 --> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8dc0aff80 con 0x7fe8dc0ad4b0 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.503+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe8e00d0c70 con 0x7fe8f0102a90 2024-04-25T13:03:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.504+0000 7fe8e6ffd700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe8dc0a3cd0 0x7fe8dc0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0101090 tx=0x7fe8d8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:25.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.504+0000 7fe8effff700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] conn(0x7fe8dc0ad4b0 0x7fe8dc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:25.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.504+0000 7fe8effff700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] conn(0x7fe8dc0ad4b0 0x7fe8dc0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:25.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.505+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== osd.8 v2:172.21.15.178:6816/2770931166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8dc0aff80 con 0x7fe8dc0ad4b0 2024-04-25T13:03:25.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.521+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 --> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe8f00635d0 con 0x7fe8dc0ad4b0 2024-04-25T13:03:25.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8ed7fa700 1 -- 172.21.15.38:0/1135740071 <== osd.8 v2:172.21.15.178:6816/2770931166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe8f00635d0 con 0x7fe8dc0ad4b0 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] conn(0x7fe8dc0ad4b0 msgr2=0x7fe8dc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] conn(0x7fe8dc0ad4b0 0x7fe8dc0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe8dc0a3cd0 msgr2=0x7fe8dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe8dc0a3cd0 0x7fe8dc0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0101090 tx=0x7fe8d8005d50 comp rx=0 tx=0).stop 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0102a90 msgr2=0x7fe8f0199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8f5d06700 1 --2- 172.21.15.38:0/1135740071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0102a90 0x7fe8f0199190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000d980 tx=0x7fe8e000dc90 comp rx=0 tx=0).stop 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.522+0000 7fe8effff700 1 -- 172.21.15.38:0/1135740071 reap_dead start 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.523+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 shutdown_connections 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.523+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 >> 172.21.15.38:0/1135740071 conn(0x7fe8f0079ba0 msgr2=0x7fe8f0101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:25.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.523+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 shutdown_connections 2024-04-25T13:03:25.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:25.523+0000 7fe8f5d06700 1 -- 172.21.15.38:0/1135740071 wait complete. 2024-04-25T13:03:27.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:27 smithi038 ceph-mon[29400]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:03:27.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:27 smithi178 ceph-mon[33233]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:03:27.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:27 smithi104 ceph-mon[32726]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:03:27.745 INFO:teuthology.orchestra.run.smithi038.stdout:34359738445 2024-04-25T13:03:27.745 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 -- 172.21.15.38:0/3794073152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c103480 msgr2=0x7f998c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 --2- 172.21.15.38:0/3794073152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c103480 0x7f998c10f980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f998400b0a0 tx=0x7f998400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 -- 172.21.15.38:0/3794073152 shutdown_connections 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 --2- 172.21.15.38:0/3794073152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c103480 0x7f998c10f980 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 --2- 172.21.15.38:0/3794073152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f998c108ab0 0x7f998c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 --2- 172.21.15.38:0/3794073152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998c102ac0 0x7f998c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 -- 172.21.15.38:0/3794073152 >> 172.21.15.38:0/3794073152 conn(0x7f998c079bf0 msgr2=0x7f998c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 -- 172.21.15.38:0/3794073152 shutdown_connections 2024-04-25T13:03:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.463+0000 7f9992825700 1 -- 172.21.15.38:0/3794073152 wait complete. 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 Processor -- start 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 -- start start 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c102ac0 0x7f998c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998c103480 0x7f998c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f998c108ab0 0x7f998c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112e30 con 0x7f998c102ac0 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112cb0 con 0x7f998c103480 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f9992825700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112fb0 con 0x7f998c108ab0 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f998bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c102ac0 0x7f998c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f998bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c102ac0 0x7f998c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54020/0 (socket says 172.21.15.38:54020) 2024-04-25T13:03:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f998bfff700 1 -- 172.21.15.38:0/390131870 learned_addr learned my addr 172.21.15.38:0/390131870 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.464+0000 7f998b7fe700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998c103480 0x7f998c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f9990dc2700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f998c108ab0 0x7f998c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998bfff700 1 -- 172.21.15.38:0/390131870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f998c108ab0 msgr2=0x7f998c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998bfff700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f998c108ab0 0x7f998c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998bfff700 1 -- 172.21.15.38:0/390131870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998c103480 msgr2=0x7f998c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998bfff700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998c103480 0x7f998c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998bfff700 1 -- 172.21.15.38:0/390131870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c1a0460 con 0x7f998c102ac0 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998b7fe700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998c103480 0x7f998c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f998bfff700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c102ac0 0x7f998c19b450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f997c00d980 tx=0x7f997c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c017070 con 0x7f998c102ac0 2024-04-25T13:03:28.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c1a0750 con 0x7f998c102ac0 2024-04-25T13:03:28.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f997c005020 con 0x7f998c102ac0 2024-04-25T13:03:28.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c009bf0 con 0x7f998c102ac0 2024-04-25T13:03:28.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.465+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1a7fd0 con 0x7f998c102ac0 2024-04-25T13:03:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.466+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f998c1012d0 con 0x7f998c102ac0 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.467+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f997c0051d0 con 0x7f998c102ac0 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.468+0000 7f99897fa700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f99740a3cd0 0x7f99740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.468+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f997c0d0820 con 0x7f998c102ac0 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.468+0000 7f998b7fe700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f99740a3cd0 0x7f99740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.468+0000 7f99897fa700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] conn(0x7f99740ad4b0 0x7f99740af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.468+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 --> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99740aff80 con 0x7f99740ad4b0 2024-04-25T13:03:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.468+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f997c0d0d00 con 0x7f998c102ac0 2024-04-25T13:03:28.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.469+0000 7f9990dc2700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] conn(0x7f99740ad4b0 0x7f99740af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:28.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.469+0000 7f998b7fe700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f99740a3cd0 0x7f99740a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f998c10fda0 tx=0x7f9980009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:28.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.469+0000 7f9990dc2700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] conn(0x7f99740ad4b0 0x7f99740af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:28.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.470+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== osd.6 v2:172.21.15.38:6818/2531974702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99740aff80 con 0x7f99740ad4b0 2024-04-25T13:03:28.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.486+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 --> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f998c1041f0 con 0x7f99740ad4b0 2024-04-25T13:03:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.486+0000 7f99897fa700 1 -- 172.21.15.38:0/390131870 <== osd.6 v2:172.21.15.38:6818/2531974702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f998c1041f0 con 0x7f99740ad4b0 2024-04-25T13:03:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] conn(0x7f99740ad4b0 msgr2=0x7f99740af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] conn(0x7f99740ad4b0 0x7f99740af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f99740a3cd0 msgr2=0x7f99740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f99740a3cd0 0x7f99740a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f998c10fda0 tx=0x7f9980009190 comp rx=0 tx=0).stop 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c102ac0 msgr2=0x7f998c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 --2- 172.21.15.38:0/390131870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f998c102ac0 0x7f998c19b450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f997c00d980 tx=0x7f997c00dc90 comp rx=0 tx=0).stop 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9990dc2700 1 -- 172.21.15.38:0/390131870 reap_dead start 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 shutdown_connections 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 >> 172.21.15.38:0/390131870 conn(0x7f998c079bf0 msgr2=0x7f998c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 shutdown_connections 2024-04-25T13:03:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:28.487+0000 7f9992825700 1 -- 172.21.15.38:0/390131870 wait complete. 2024-04-25T13:03:28.868 INFO:teuthology.orchestra.run.smithi038.stdout:292057776159 2024-04-25T13:03:28.868 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:03:29.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:29 smithi038 ceph-mon[29400]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:29.293 INFO:teuthology.orchestra.run.smithi038.stdout:111669149758 2024-04-25T13:03:29.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:03:29.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 -- 172.21.15.38:0/1763113558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f0109c80 msgr2=0x7f12f010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 --2- 172.21.15.38:0/1763113558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f0109c80 0x7f12f010dcd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12e40099c0 tx=0x7f12e4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 -- 172.21.15.38:0/1763113558 shutdown_connections 2024-04-25T13:03:29.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 --2- 172.21.15.38:0/1763113558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12f01092d0 0x7f12f01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 --2- 172.21.15.38:0/1763113558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f010e3b0 0x7f12f0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 --2- 172.21.15.38:0/1763113558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f0109c80 0x7f12f010dcd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.481+0000 7f12f626b700 1 -- 172.21.15.38:0/1763113558 >> 172.21.15.38:0/1763113558 conn(0x7f12f0076210 msgr2=0x7f12f0078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:29.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.482+0000 7f12f626b700 1 -- 172.21.15.38:0/1763113558 shutdown_connections 2024-04-25T13:03:29.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.482+0000 7f12f626b700 1 -- 172.21.15.38:0/1763113558 wait complete. 2024-04-25T13:03:29.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.482+0000 7f12f626b700 1 Processor -- start 2024-04-25T13:03:29.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.482+0000 7f12f626b700 1 -- start start 2024-04-25T13:03:29.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f626b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f01092d0 0x7f12f0074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f626b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f0109c80 0x7f12f00746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f626b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12f010e3b0 0x7f12f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f626b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0119360 con 0x7f12f01092d0 2024-04-25T13:03:29.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f626b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01191e0 con 0x7f12f010e3b0 2024-04-25T13:03:29.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f626b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01194e0 con 0x7f12f0109c80 2024-04-25T13:03:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f01092d0 0x7f12f0074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f0109c80 0x7f12f00746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f01092d0 0x7f12f0074170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54038/0 (socket says 172.21.15.38:54038) 2024-04-25T13:03:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12effff700 1 -- 172.21.15.38:0/603431734 learned_addr learned my addr 172.21.15.38:0/603431734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.483+0000 7f12f4808700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12f010e3b0 0x7f12f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12effff700 1 -- 172.21.15.38:0/603431734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f0109c80 msgr2=0x7f12f00746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12effff700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f0109c80 0x7f12f00746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12effff700 1 -- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12f010e3b0 msgr2=0x7f12f006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12effff700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12f010e3b0 0x7f12f006d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12effff700 1 -- 172.21.15.38:0/603431734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f006dd50 con 0x7f12f01092d0 2024-04-25T13:03:29.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12f4808700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12f010e3b0 0x7f12f006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:03:29.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12effff700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f01092d0 0x7f12f0074170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c920 tx=0x7f12e000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:29.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0004a30 con 0x7f12f01092d0 2024-04-25T13:03:29.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f006e040 con 0x7f12f01092d0 2024-04-25T13:03:29.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.484+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f0071f30 con 0x7f12f01092d0 2024-04-25T13:03:29.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.485+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f12e00054f0 con 0x7f12f01092d0 2024-04-25T13:03:29.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.485+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0007d40 con 0x7f12f01092d0 2024-04-25T13:03:29.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:29 smithi178 ceph-mon[33233]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:29.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.486+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f12dc001030 con 0x7f12f01092d0 2024-04-25T13:03:29.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.487+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f12e0005020 con 0x7f12f01092d0 2024-04-25T13:03:29.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.488+0000 7f12ed7fa700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f12d80a3e20 0x7f12d80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12ef7fe700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f12d80a3e20 0x7f12d80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12e00d0270 con 0x7f12f01092d0 2024-04-25T13:03:29.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12ed7fa700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] conn(0x7f12d80ad600 0x7f12d80afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 --> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12e0012ec0 con 0x7f12d80ad600 2024-04-25T13:03:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f12e00d0750 con 0x7f12f01092d0 2024-04-25T13:03:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12ef7fe700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f12d80a3e20 0x7f12d80a62e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12e4006010 tx=0x7f12e4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.489+0000 7f12f4808700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] conn(0x7f12d80ad600 0x7f12d80afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.490+0000 7f12f4808700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] conn(0x7f12d80ad600 0x7f12d80afa20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:29.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.490+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== osd.7 v2:172.21.15.104:6816/3955477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12e0012ec0 con 0x7f12d80ad600 2024-04-25T13:03:29.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.519+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 --> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f12dc002e60 con 0x7f12d80ad600 2024-04-25T13:03:29.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.519+0000 7f12ed7fa700 1 -- 172.21.15.38:0/603431734 <== osd.7 v2:172.21.15.104:6816/3955477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f12dc002e60 con 0x7f12d80ad600 2024-04-25T13:03:29.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] conn(0x7f12d80ad600 msgr2=0x7f12d80afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] conn(0x7f12d80ad600 0x7f12d80afa20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f12d80a3e20 msgr2=0x7f12d80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f12d80a3e20 0x7f12d80a62e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12e4006010 tx=0x7f12e4005c60 comp rx=0 tx=0).stop 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f01092d0 msgr2=0x7f12f0074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 --2- 172.21.15.38:0/603431734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12f01092d0 0x7f12f0074170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c920 tx=0x7f12e000cc30 comp rx=0 tx=0).stop 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f4808700 1 -- 172.21.15.38:0/603431734 reap_dead start 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 shutdown_connections 2024-04-25T13:03:29.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 >> 172.21.15.38:0/603431734 conn(0x7f12f0076210 msgr2=0x7f12f00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:29.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 shutdown_connections 2024-04-25T13:03:29.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.520+0000 7f12f626b700 1 -- 172.21.15.38:0/603431734 wait complete. 2024-04-25T13:03:29.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:29 smithi104 ceph-mon[32726]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 -- 172.21.15.38:0/968534970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 msgr2=0x7f1c6c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 --2- 172.21.15.38:0/968534970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c10e7f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009990 tx=0x7f1c5c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 -- 172.21.15.38:0/968534970 shutdown_connections 2024-04-25T13:03:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 --2- 172.21.15.38:0/968534970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c6c107990 0x7f1c6c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 --2- 172.21.15.38:0/968534970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c10e7f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 --2- 172.21.15.38:0/968534970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c6c101940 0x7f1c6c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 -- 172.21.15.38:0/968534970 >> 172.21.15.38:0/968534970 conn(0x7f1c6c0788f0 msgr2=0x7f1c6c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 -- 172.21.15.38:0/968534970 shutdown_connections 2024-04-25T13:03:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.701+0000 7f1c73b27700 1 -- 172.21.15.38:0/968534970 wait complete. 2024-04-25T13:03:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 Processor -- start 2024-04-25T13:03:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 -- start start 2024-04-25T13:03:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c6c101940 0x7f1c6c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c6c107990 0x7f1c6c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c111c10 con 0x7f1c6c102300 2024-04-25T13:03:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c111a90 con 0x7f1c6c101940 2024-04-25T13:03:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c73b27700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c111d90 con 0x7f1c6c107990 2024-04-25T13:03:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60116/0 (socket says 172.21.15.38:60116) 2024-04-25T13:03:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 -- 172.21.15.38:0/3286455098 learned_addr learned my addr 172.21.15.38:0/3286455098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c720c4700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c6c107990 0x7f1c6c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c718c3700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c6c101940 0x7f1c6c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 -- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c6c107990 msgr2=0x7f1c6c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c6c107990 0x7f1c6c19da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 -- 172.21.15.38:0/3286455098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c6c101940 msgr2=0x7f1c6c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c6c101940 0x7f1c6c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 -- 172.21.15.38:0/3286455098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19e1d0 con 0x7f1c6c102300 2024-04-25T13:03:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c718c3700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c6c101940 0x7f1c6c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c710c2700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c199700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c920 tx=0x7f1c6000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.702+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60007b00 con 0x7f1c6c102300 2024-04-25T13:03:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.703+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19e4c0 con 0x7f1c6c102300 2024-04-25T13:03:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.703+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1c60004d00 con 0x7f1c6c102300 2024-04-25T13:03:29.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.703+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60005650 con 0x7f1c6c102300 2024-04-25T13:03:29.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.703+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a5d40 con 0x7f1c6c102300 2024-04-25T13:03:29.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.703+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c6c063150 con 0x7f1c6c102300 2024-04-25T13:03:29.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1c60007ca0 con 0x7f1c6c102300 2024-04-25T13:03:29.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c6affd700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1c540a3cd0 0x7f1c540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c718c3700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1c540a3cd0 0x7f1c540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1c600d0340 con 0x7f1c6c102300 2024-04-25T13:03:29.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c6affd700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] conn(0x7f1c540ad4b0 0x7f1c540af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:29.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 --> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c60023ef0 con 0x7f1c540ad4b0 2024-04-25T13:03:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.705+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1c600d0820 con 0x7f1c6c102300 2024-04-25T13:03:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.706+0000 7f1c718c3700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1c540a3cd0 0x7f1c540a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c640093a0 tx=0x7f1c6400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.706+0000 7f1c720c4700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] conn(0x7f1c540ad4b0 0x7f1c540af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.706+0000 7f1c720c4700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] conn(0x7f1c540ad4b0 0x7f1c540af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.707+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== osd.5 v2:172.21.15.178:6808/1085104409 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c60023ef0 con 0x7f1c540ad4b0 2024-04-25T13:03:29.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.724+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 --> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c6c0635d0 con 0x7f1c540ad4b0 2024-04-25T13:03:29.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.725+0000 7f1c6affd700 1 -- 172.21.15.38:0/3286455098 <== osd.5 v2:172.21.15.178:6808/1085104409 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1c6c0635d0 con 0x7f1c540ad4b0 2024-04-25T13:03:29.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.725+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] conn(0x7f1c540ad4b0 msgr2=0x7f1c540af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.725+0000 7f1c73b27700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] conn(0x7f1c540ad4b0 0x7f1c540af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:29.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1c540a3cd0 msgr2=0x7f1c540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1c540a3cd0 0x7f1c540a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c640093a0 tx=0x7f1c6400b040 comp rx=0 tx=0).stop 2024-04-25T13:03:29.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 msgr2=0x7f1c6c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:29.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 --2- 172.21.15.38:0/3286455098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c102300 0x7f1c6c199700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c920 tx=0x7f1c6000cc30 comp rx=0 tx=0).stop 2024-04-25T13:03:29.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c720c4700 1 -- 172.21.15.38:0/3286455098 reap_dead start 2024-04-25T13:03:29.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 shutdown_connections 2024-04-25T13:03:29.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 >> 172.21.15.38:0/3286455098 conn(0x7f1c6c0788f0 msgr2=0x7f1c6c06aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 shutdown_connections 2024-04-25T13:03:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:29.726+0000 7f1c73b27700 1 -- 172.21.15.38:0/3286455098 wait complete. 2024-04-25T13:03:30.399 INFO:teuthology.orchestra.run.smithi038.stdout:146028888121 2024-04-25T13:03:30.399 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:03:30.534 INFO:teuthology.orchestra.run.smithi038.stdout:257698037796 2024-04-25T13:03:30.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:03:30.942 INFO:teuthology.orchestra.run.smithi038.stdout:55834574921 2024-04-25T13:03:30.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:03:31.161 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:31 smithi038 ceph-mon[29400]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:03:31.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:31 smithi178 ceph-mon[33233]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:03:31.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:31 smithi104 ceph-mon[32726]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:03:32.075 INFO:teuthology.orchestra.run.smithi038.stdout:223338299435 2024-04-25T13:03:32.075 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:03:32.529 INFO:teuthology.orchestra.run.smithi038.stdout:201863462959 2024-04-25T13:03:32.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:03:32.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.735+0000 7fe92c486700 1 -- 172.21.15.38:0/630098503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924108ad0 msgr2=0x7fe924108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.735+0000 7fe92c486700 1 --2- 172.21.15.38:0/630098503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924108ad0 0x7fe924108eb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe920009990 tx=0x7fe920009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.736+0000 7fe92c486700 1 -- 172.21.15.38:0/630098503 shutdown_connections 2024-04-25T13:03:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.736+0000 7fe92c486700 1 --2- 172.21.15.38:0/630098503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9241019d0 0x7fe92410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.736+0000 7fe92c486700 1 --2- 172.21.15.38:0/630098503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924108ad0 0x7fe924108eb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.736+0000 7fe92c486700 1 --2- 172.21.15.38:0/630098503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe924101030 0x7fe924101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.736+0000 7fe92c486700 1 -- 172.21.15.38:0/630098503 >> 172.21.15.38:0/630098503 conn(0x7fe924077fb0 msgr2=0x7fe9240ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:32.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.738+0000 7fe92c486700 1 -- 172.21.15.38:0/630098503 shutdown_connections 2024-04-25T13:03:32.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.738+0000 7fe92c486700 1 -- 172.21.15.38:0/630098503 wait complete. 2024-04-25T13:03:32.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.739+0000 7fe92c486700 1 Processor -- start 2024-04-25T13:03:32.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.739+0000 7fe92c486700 1 -- start start 2024-04-25T13:03:32.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92c486700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924101030 0x7fe924199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:32.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92c486700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9241019d0 0x7fe9241997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92c486700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe924108ad0 0x7fe92419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92c486700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe924112da0 con 0x7fe924101030 2024-04-25T13:03:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92c486700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe924112c20 con 0x7fe924108ad0 2024-04-25T13:03:32.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92c486700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe924112f20 con 0x7fe9241019d0 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92a222700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924101030 0x7fe924199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92a222700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924101030 0x7fe924199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60126/0 (socket says 172.21.15.38:60126) 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92a222700 1 -- 172.21.15.38:0/1327827346 learned_addr learned my addr 172.21.15.38:0/1327827346 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.740+0000 7fe92aa23700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe924108ad0 0x7fe92419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe929a21700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9241019d0 0x7fe9241997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe92aa23700 1 -- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9241019d0 msgr2=0x7fe9241997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe92aa23700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9241019d0 0x7fe9241997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe92aa23700 1 -- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924101030 msgr2=0x7fe924199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:32.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe92aa23700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924101030 0x7fe924199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe92aa23700 1 -- 172.21.15.38:0/1327827346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92419e280 con 0x7fe924108ad0 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.741+0000 7fe92aa23700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe924108ad0 0x7fe92419db40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00be00 tx=0x7fe91c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.742+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c019070 con 0x7fe924108ad0 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.742+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe91c00cc10 con 0x7fe924108ad0 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.742+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c012690 con 0x7fe924108ad0 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.742+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92419e570 con 0x7fe924108ad0 2024-04-25T13:03:32.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.742+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9241a5df0 con 0x7fe924108ad0 2024-04-25T13:03:32.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.743+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe908001030 con 0x7fe924108ad0 2024-04-25T13:03:32.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.744+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe91c01c070 con 0x7fe924108ad0 2024-04-25T13:03:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.745+0000 7fe91b7fe700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe9100a3d50 0x7fe9100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.746+0000 7fe92a222700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe9100a3d50 0x7fe9100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.746+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe91c0cf7c0 con 0x7fe924108ad0 2024-04-25T13:03:32.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.746+0000 7fe91b7fe700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] conn(0x7fe9100ad530 0x7fe9100af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.746+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 --> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9100b0000 con 0x7fe9100ad530 2024-04-25T13:03:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.746+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe91c0d6050 con 0x7fe924108ad0 2024-04-25T13:03:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.746+0000 7fe929a21700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] conn(0x7fe9100ad530 0x7fe9100af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.747+0000 7fe92a222700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe9100a3d50 0x7fe9100a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe920009750 tx=0x7fe92000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:32.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.747+0000 7fe929a21700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] conn(0x7fe9100ad530 0x7fe9100af950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:32.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.748+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== osd.2 v2:172.21.15.178:6800/760031576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9100b0000 con 0x7fe9100ad530 2024-04-25T13:03:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.770+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 --> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe908002e60 con 0x7fe9100ad530 2024-04-25T13:03:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe91b7fe700 1 -- 172.21.15.38:0/1327827346 <== osd.2 v2:172.21.15.178:6800/760031576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe908002e60 con 0x7fe9100ad530 2024-04-25T13:03:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] conn(0x7fe9100ad530 msgr2=0x7fe9100af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92c486700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] conn(0x7fe9100ad530 0x7fe9100af950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe9100a3d50 msgr2=0x7fe9100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92c486700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe9100a3d50 0x7fe9100a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe920009750 tx=0x7fe92000b260 comp rx=0 tx=0).stop 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe924108ad0 msgr2=0x7fe92419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92c486700 1 --2- 172.21.15.38:0/1327827346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe924108ad0 0x7fe92419db40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00be00 tx=0x7fe91c009ed0 comp rx=0 tx=0).stop 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.771+0000 7fe92aa23700 1 -- 172.21.15.38:0/1327827346 reap_dead start 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.772+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 shutdown_connections 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.772+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 >> 172.21.15.38:0/1327827346 conn(0x7fe924077fb0 msgr2=0x7fe9240ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.772+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 shutdown_connections 2024-04-25T13:03:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:32.772+0000 7fe92c486700 1 -- 172.21.15.38:0/1327827346 wait complete. 2024-04-25T13:03:33.159 INFO:teuthology.orchestra.run.smithi038.stdout:167503724597 2024-04-25T13:03:33.160 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:03:33.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:33 smithi178 ceph-mon[33233]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:03:33.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:33 smithi038 ceph-mon[29400]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:03:33.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:33 smithi104 ceph-mon[32726]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:03:33.833 INFO:teuthology.orchestra.run.smithi038.stdout:77309411397 2024-04-25T13:03:33.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:03:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:33.882 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:35.461 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:35 smithi038 ceph-mon[29400]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:35.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:35 smithi178 ceph-mon[33233]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:35 smithi104 ceph-mon[32726]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T13:03:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T13:03:36.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:03:36.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:36 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T13:03:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T13:03:36.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:03:36.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:36 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:03:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:03:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:03:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T13:03:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T13:03:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T13:03:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T13:03:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T13:03:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:03:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:36 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:03:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T13:03:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:03:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:03:37.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:37 smithi038 ceph-mon[29400]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T13:03:37.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T13:03:37.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:03:37.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:03:37.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:37 smithi178 ceph-mon[33233]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:03:37.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:03:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:37 smithi104 ceph-mon[32726]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:03:38.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:38 smithi038 ceph-mon[29400]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:03:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:38 smithi178 ceph-mon[33233]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:03:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:38 smithi104 ceph-mon[32726]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:03:39.411 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:39 smithi038 ceph-mon[29400]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-25T13:03:39.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:39 smithi178 ceph-mon[33233]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-25T13:03:39.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:39 smithi104 ceph-mon[32726]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-25T13:03:41.312 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:41 smithi038 ceph-mon[29400]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T13:03:41.312 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2760210632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:03:41.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:41 smithi178 ceph-mon[33233]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T13:03:41.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2760210632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:03:41.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:41 smithi104 ceph-mon[32726]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T13:03:41.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2760210632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 -- 172.21.15.38:0/2489054920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c102ac0 msgr2=0x7f1d8c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 --2- 172.21.15.38:0/2489054920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c102ac0 0x7f1d8c102f40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009990 tx=0x7f1d80009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 -- 172.21.15.38:0/2489054920 shutdown_connections 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 --2- 172.21.15.38:0/2489054920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c103480 0x7f1d8c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 --2- 172.21.15.38:0/2489054920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 --2- 172.21.15.38:0/2489054920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c102ac0 0x7f1d8c102f40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 -- 172.21.15.38:0/2489054920 >> 172.21.15.38:0/2489054920 conn(0x7f1d8c079bf0 msgr2=0x7f1d8c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 -- 172.21.15.38:0/2489054920 shutdown_connections 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.418+0000 7f1d90e83700 1 -- 172.21.15.38:0/2489054920 wait complete. 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 Processor -- start 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 -- start start 2024-04-25T13:03:42.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d8c102ac0 0x7f1d8c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c103480 0x7f1d8c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c112db0 con 0x7f1d8c108ab0 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c112c30 con 0x7f1d8c103480 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d90e83700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c112f30 con 0x7f1d8c102ac0 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d8ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d8ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c0757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51300/0 (socket says 172.21.15.38:51300) 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.419+0000 7f1d8ad9d700 1 -- 172.21.15.38:0/3070973334 learned_addr learned my addr 172.21.15.38:0/3070973334 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8a59c700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d8c102ac0 0x7f1d8c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d89d9b700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c103480 0x7f1d8c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8ad9d700 1 -- 172.21.15.38:0/3070973334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d8c102ac0 msgr2=0x7f1d8c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8ad9d700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d8c102ac0 0x7f1d8c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8ad9d700 1 -- 172.21.15.38:0/3070973334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c103480 msgr2=0x7f1d8c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8ad9d700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c103480 0x7f1d8c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8ad9d700 1 -- 172.21.15.38:0/3070973334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c1a7770 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d8ad9d700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c0757c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400ada0 tx=0x7f1d8400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017020 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c1a7a60 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1d8400ce50 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84005560 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.420+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1a7f50 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.421+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c069730 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.422+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1d84004ad0 con 0x7f1d8c108ab0 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.423+0000 7f1d7b7fe700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1d740a3cd0 0x7f1d740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.423+0000 7f1d8a59c700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1d740a3cd0 0x7f1d740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:42.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.423+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1d840cfd80 con 0x7f1d8c108ab0 2024-04-25T13:03:42.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.424+0000 7f1d8a59c700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1d740a3cd0 0x7f1d740a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005fd0 tx=0x7f1d7c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:42.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.428+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d84021020 con 0x7f1d8c108ab0 2024-04-25T13:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.642+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1d8c02df80 con 0x7f1d8c108ab0 2024-04-25T13:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.643+0000 7f1d7b7fe700 1 -- 172.21.15.38:0/3070973334 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d8409c690 con 0x7f1d8c108ab0 2024-04-25T13:03:42.641 INFO:teuthology.orchestra.run.smithi038.stdout:292057776163 2024-04-25T13:03:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1d740a3cd0 msgr2=0x7f1d740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1d740a3cd0 0x7f1d740a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005fd0 tx=0x7f1d7c005db0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 msgr2=0x7f1d8c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c0757c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400ada0 tx=0x7f1d8400c330 comp rx=0 tx=0).stop 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 shutdown_connections 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1d740a3cd0 0x7f1d740a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d8c108ab0 0x7f1d8c0757c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d8c102ac0 0x7f1d8c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 --2- 172.21.15.38:0/3070973334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c103480 0x7f1d8c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 >> 172.21.15.38:0/3070973334 conn(0x7f1d8c079bf0 msgr2=0x7f1d8c100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 shutdown_connections 2024-04-25T13:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:42.645+0000 7f1d90e83700 1 -- 172.21.15.38:0/3070973334 wait complete. 2024-04-25T13:03:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:43 smithi038 ceph-mon[29400]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T13:03:43.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:43 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3070973334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:03:43.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776163 for osd.9 2024-04-25T13:03:43.384 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:43.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:43 smithi178 ceph-mon[33233]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T13:03:43.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:43 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3070973334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:03:43.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:43 smithi104 ceph-mon[32726]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T13:03:43.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:43 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3070973334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:03:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 -- 172.21.15.38:0/3332193723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806b0f0 msgr2=0x7fe3e810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 --2- 172.21.15.38:0/3332193723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806b0f0 0x7fe3e810f900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000b0a0 tx=0x7fe3e000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 -- 172.21.15.38:0/3332193723 shutdown_connections 2024-04-25T13:03:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 --2- 172.21.15.38:0/3332193723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806b0f0 0x7fe3e810f900 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 --2- 172.21.15.38:0/3332193723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e8108a30 0x7fe3e8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 --2- 172.21.15.38:0/3332193723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806a750 0x7fe3e806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 -- 172.21.15.38:0/3332193723 >> 172.21.15.38:0/3332193723 conn(0x7fe3e80788f0 msgr2=0x7fe3e8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 -- 172.21.15.38:0/3332193723 shutdown_connections 2024-04-25T13:03:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 -- 172.21.15.38:0/3332193723 wait complete. 2024-04-25T13:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 Processor -- start 2024-04-25T13:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.987+0000 7fe3efe99700 1 -- start start 2024-04-25T13:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 0x7fe3e819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806b0f0 0x7fe3e819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e8108a30 0x7fe3e819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8112d90 con 0x7fe3e806a750 2024-04-25T13:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8112c10 con 0x7fe3e806b0f0 2024-04-25T13:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8112f10 con 0x7fe3e8108a30 2024-04-25T13:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 0x7fe3e819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 0x7fe3e819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51328/0 (socket says 172.21.15.38:51328) 2024-04-25T13:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 -- 172.21.15.38:0/2421565014 learned_addr learned my addr 172.21.15.38:0/2421565014 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3ed434700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806b0f0 0x7fe3e819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3ee436700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e8108a30 0x7fe3e819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 -- 172.21.15.38:0/2421565014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e8108a30 msgr2=0x7fe3e819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e8108a30 0x7fe3e819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 -- 172.21.15.38:0/2421565014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806b0f0 msgr2=0x7fe3e819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806b0f0 0x7fe3e819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 -- 172.21.15.38:0/2421565014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81a04a0 con 0x7fe3e806a750 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3ed434700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806b0f0 0x7fe3e819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3edc35700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 0x7fe3e819b4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c920 tx=0x7fe3e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4007b00 con 0x7fe3e806a750 2024-04-25T13:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe3e4004d00 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e81a0790 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.988+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400f450 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.989+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e81a8010 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.989+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e8069730 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.991+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe3e4007ca0 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.991+0000 7fe3deffd700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe3d40a3d50 0x7fe3d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.991+0000 7fe3ed434700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe3d40a3d50 0x7fe3d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.991+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe3e40d0510 con 0x7fe3e806a750 2024-04-25T13:03:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.992+0000 7fe3ed434700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe3d40a3d50 0x7fe3d40a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8078f30 tx=0x7fe3d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:43.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:43.996+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3e4014030 con 0x7fe3e806a750 2024-04-25T13:03:44.121 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:44.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.213+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe3e802df80 con 0x7fe3e806a750 2024-04-25T13:03:44.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.213+0000 7fe3deffd700 1 -- 172.21.15.38:0/2421565014 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe3e409cd90 con 0x7fe3e806a750 2024-04-25T13:03:44.212 INFO:teuthology.orchestra.run.smithi038.stdout:34359738449 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe3d40a3d50 msgr2=0x7fe3d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe3d40a3d50 0x7fe3d40a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8078f30 tx=0x7fe3d8008040 comp rx=0 tx=0).stop 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 msgr2=0x7fe3e819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 0x7fe3e819b4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c920 tx=0x7fe3e400cce0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 shutdown_connections 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe3d40a3d50 0x7fe3d40a6210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e8108a30 0x7fe3e819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3e806a750 0x7fe3e819b4e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 --2- 172.21.15.38:0/2421565014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3e806b0f0 0x7fe3e819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.215+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 >> 172.21.15.38:0/2421565014 conn(0x7fe3e80788f0 msgr2=0x7fe3e810df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.216+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 shutdown_connections 2024-04-25T13:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.216+0000 7fe3efe99700 1 -- 172.21.15.38:0/2421565014 wait complete. 2024-04-25T13:03:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 -- 172.21.15.38:0/667087127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006b0f0 msgr2=0x7fdf5010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 --2- 172.21.15.38:0/667087127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006b0f0 0x7fdf5010f900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b0a0 tx=0x7fdf4800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 -- 172.21.15.38:0/667087127 shutdown_connections 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 --2- 172.21.15.38:0/667087127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006b0f0 0x7fdf5010f900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 --2- 172.21.15.38:0/667087127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf50108a30 0x7fdf50108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 --2- 172.21.15.38:0/667087127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006a750 0x7fdf5006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 -- 172.21.15.38:0/667087127 >> 172.21.15.38:0/667087127 conn(0x7fdf500788f0 msgr2=0x7fdf50078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 -- 172.21.15.38:0/667087127 shutdown_connections 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.226+0000 7fdf570ba700 1 -- 172.21.15.38:0/667087127 wait complete. 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 Processor -- start 2024-04-25T13:03:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 -- start start 2024-04-25T13:03:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 0x7fdf501991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006b0f0 0x7fdf50199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf50108a30 0x7fdf5019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50112d90 con 0x7fdf5006a750 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50112c10 con 0x7fdf5006b0f0 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf570ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50112f10 con 0x7fdf50108a30 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf54e56700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 0x7fdf501991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf54e56700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 0x7fdf501991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51340/0 (socket says 172.21.15.38:51340) 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.227+0000 7fdf54e56700 1 -- 172.21.15.38:0/2566058742 learned_addr learned my addr 172.21.15.38:0/2566058742 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006b0f0 0x7fdf50199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf54e56700 1 -- 172.21.15.38:0/2566058742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf50108a30 msgr2=0x7fdf5019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf54e56700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf50108a30 0x7fdf5019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf54e56700 1 -- 172.21.15.38:0/2566058742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006b0f0 msgr2=0x7fdf50199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf54e56700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006b0f0 0x7fdf50199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf54e56700 1 -- 172.21.15.38:0/2566058742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5019e1a0 con 0x7fdf5006a750 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006b0f0 0x7fdf50199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.228+0000 7fdf54e56700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 0x7fdf501991e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e9c0 tx=0x7fdf4400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.229+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44013be0 con 0x7fdf5006a750 2024-04-25T13:03:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.229+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdf44013d80 con 0x7fdf5006a750 2024-04-25T13:03:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.229+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44005020 con 0x7fdf5006a750 2024-04-25T13:03:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.229+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5019e490 con 0x7fdf5006a750 2024-04-25T13:03:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.229+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf50101940 con 0x7fdf5006a750 2024-04-25T13:03:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.230+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf50069730 con 0x7fdf5006a750 2024-04-25T13:03:44.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.233+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdf440051c0 con 0x7fdf5006a750 2024-04-25T13:03:44.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.234+0000 7fdf4dffb700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf400a3e20 0x7fdf400a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:44.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.234+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf400a3e20 0x7fdf400a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:44.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.234+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdf440d4b50 con 0x7fdf5006a750 2024-04-25T13:03:44.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.235+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf400a3e20 0x7fdf400a62e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf50078f60 tx=0x7fdf3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.237+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf440a1560 con 0x7fdf5006a750 2024-04-25T13:03:44.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.451+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fdf5002df80 con 0x7fdf5006a750 2024-04-25T13:03:44.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.452+0000 7fdf4dffb700 1 -- 172.21.15.38:0/2566058742 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdf440a0e60 con 0x7fdf5006a750 2024-04-25T13:03:44.450 INFO:teuthology.orchestra.run.smithi038.stdout:313532612637 2024-04-25T13:03:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf400a3e20 msgr2=0x7fdf400a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf400a3e20 0x7fdf400a62e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf50078f60 tx=0x7fdf3c009260 comp rx=0 tx=0).stop 2024-04-25T13:03:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 msgr2=0x7fdf501991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 0x7fdf501991e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e9c0 tx=0x7fdf4400ed80 comp rx=0 tx=0).stop 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 shutdown_connections 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf400a3e20 0x7fdf400a62e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf50108a30 0x7fdf5019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf5006a750 0x7fdf501991e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 --2- 172.21.15.38:0/2566058742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf5006b0f0 0x7fdf50199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 >> 172.21.15.38:0/2566058742 conn(0x7fdf500788f0 msgr2=0x7fdf500ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 shutdown_connections 2024-04-25T13:03:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:44.454+0000 7fdf570ba700 1 -- 172.21.15.38:0/2566058742 wait complete. 2024-04-25T13:03:45.271 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:45 smithi038 ceph-mon[29400]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 37 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:03:45.271 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2421565014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:03:45.271 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:45 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2566058742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:03:45.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:45 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:45.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:45 smithi178 ceph-mon[33233]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 37 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:03:45.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:45 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2421565014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:03:45.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:45 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2566058742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:03:45.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:45 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:45.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:45 smithi104 ceph-mon[32726]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 37 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:03:45.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:45 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2421565014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:03:45.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:45 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2566058742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:03:45.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:45 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:03:45.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612637 for osd.10 2024-04-25T13:03:45.948 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:46.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738449 for osd.0 2024-04-25T13:03:46.823 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 -- 172.21.15.38:0/2177908198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 msgr2=0x7f0e4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/2177908198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e4006b750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0e300099c0 tx=0x7f0e30009cd0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 -- 172.21.15.38:0/2177908198 shutdown_connections 2024-04-25T13:03:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/2177908198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e40107e10 0x7f0e4010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/2177908198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e4006b750 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/2177908198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4006bd20 0x7f0e401076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.006+0000 7f0e44fe3700 1 -- 172.21.15.38:0/2177908198 >> 172.21.15.38:0/2177908198 conn(0x7f0e400788f0 msgr2=0x7f0e40078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.007+0000 7f0e44fe3700 1 -- 172.21.15.38:0/2177908198 shutdown_connections 2024-04-25T13:03:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.007+0000 7f0e44fe3700 1 -- 172.21.15.38:0/2177908198 wait complete. 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 Processor -- start 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 -- start start 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e401991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4006bd20 0x7f0e40199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e40107e10 0x7f0e4019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40112da0 con 0x7f0e4006b370 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40112c20 con 0x7f0e4006bd20 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e44fe3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40112f20 con 0x7f0e40107e10 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e401991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e401991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51344/0 (socket says 172.21.15.38:51344) 2024-04-25T13:03:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 -- 172.21.15.38:0/750192667 learned_addr learned my addr 172.21.15.38:0/750192667 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3dd9b700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4006bd20 0x7f0e40199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3ed9d700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e40107e10 0x7f0e4019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 -- 172.21.15.38:0/750192667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e40107e10 msgr2=0x7f0e4019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e40107e10 0x7f0e4019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 -- 172.21.15.38:0/750192667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4006bd20 msgr2=0x7f0e40199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4006bd20 0x7f0e40199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.008+0000 7f0e3e59c700 1 -- 172.21.15.38:0/750192667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4019e1b0 con 0x7f0e4006b370 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.009+0000 7f0e3e59c700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e401991f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009990 tx=0x7f0e30005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.009+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30026070 con 0x7f0e4006b370 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.009+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0e30005500 con 0x7f0e4006b370 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.009+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4019e440 con 0x7f0e4006b370 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.009+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30004960 con 0x7f0e4006b370 2024-04-25T13:03:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.009+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e401a5d20 con 0x7f0e4006b370 2024-04-25T13:03:47.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.010+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4010b050 con 0x7f0e4006b370 2024-04-25T13:03:47.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.011+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e30029070 con 0x7f0e4006b370 2024-04-25T13:03:47.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.011+0000 7f0e2f7fe700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0e280a3cd0 0x7f0e280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.012+0000 7f0e3dd9b700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0e280a3cd0 0x7f0e280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.012+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0e300d0de0 con 0x7f0e4006b370 2024-04-25T13:03:47.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.012+0000 7f0e3dd9b700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0e280a3cd0 0x7f0e280a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e40101890 tx=0x7f0e3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:47.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.016+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e3005dff0 con 0x7f0e4006b370 2024-04-25T13:03:47.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.242+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0e4019a500 con 0x7f0e4006b370 2024-04-25T13:03:47.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.242+0000 7f0e2f7fe700 1 -- 172.21.15.38:0/750192667 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0e30013070 con 0x7f0e4006b370 2024-04-25T13:03:47.241 INFO:teuthology.orchestra.run.smithi038.stdout:223338299438 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0e280a3cd0 msgr2=0x7f0e280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0e280a3cd0 0x7f0e280a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e40101890 tx=0x7f0e3400b410 comp rx=0 tx=0).stop 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 msgr2=0x7f0e401991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e401991f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009990 tx=0x7f0e30005e80 comp rx=0 tx=0).stop 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 shutdown_connections 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f0e280a3cd0 0x7f0e280a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e40107e10 0x7f0e4019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4006b370 0x7f0e401991f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 --2- 172.21.15.38:0/750192667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4006bd20 0x7f0e40199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.244+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 >> 172.21.15.38:0/750192667 conn(0x7f0e400788f0 msgr2=0x7f0e400fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.245+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 shutdown_connections 2024-04-25T13:03:47.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.245+0000 7f0e44fe3700 1 -- 172.21.15.38:0/750192667 wait complete. 2024-04-25T13:03:47.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:47 smithi038 ceph-mon[29400]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 32 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:03:47.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:47 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:47 smithi178 ceph-mon[33233]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 32 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:03:47.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:47 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:47 smithi104 ceph-mon[32726]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 32 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T13:03:47.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:47 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:03:47.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 -- 172.21.15.38:0/4160859799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4102af0 msgr2=0x7efde4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 --2- 172.21.15.38:0/4160859799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4102af0 0x7efde4102f70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efdd40099c0 tx=0x7efdd4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 -- 172.21.15.38:0/4160859799 shutdown_connections 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 --2- 172.21.15.38:0/4160859799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efde4107940 0x7efde4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 --2- 172.21.15.38:0/4160859799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efde41034b0 0x7efde410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 --2- 172.21.15.38:0/4160859799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4102af0 0x7efde4102f70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 -- 172.21.15.38:0/4160859799 >> 172.21.15.38:0/4160859799 conn(0x7efde4079ba0 msgr2=0x7efde4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 -- 172.21.15.38:0/4160859799 shutdown_connections 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.867+0000 7efdebc77700 1 -- 172.21.15.38:0/4160859799 wait complete. 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 Processor -- start 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 -- start start 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efde4102af0 0x7efde4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efde41034b0 0x7efde4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 0x7efde419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efde4112dd0 con 0x7efde4107940 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efde4112c50 con 0x7efde4102af0 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdebc77700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efde4112f50 con 0x7efde41034b0 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdea214700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 0x7efde419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdea214700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 0x7efde419da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51348/0 (socket says 172.21.15.38:51348) 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efde9a13700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efde4102af0 0x7efde4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efdea214700 1 -- 172.21.15.38:0/3247304713 learned_addr learned my addr 172.21.15.38:0/3247304713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.868+0000 7efde9212700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efde41034b0 0x7efde4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdea214700 1 -- 172.21.15.38:0/3247304713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efde41034b0 msgr2=0x7efde4199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdea214700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efde41034b0 0x7efde4199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdea214700 1 -- 172.21.15.38:0/3247304713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efde4102af0 msgr2=0x7efde4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdea214700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efde4102af0 0x7efde4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:47.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdea214700 1 -- 172.21.15.38:0/3247304713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde419e150 con 0x7efde4107940 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdea214700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 0x7efde419da10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efddc00cca0 tx=0x7efddc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc016020 con 0x7efde4107940 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde419e440 con 0x7efde4107940 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde41a5cc0 con 0x7efde4107940 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efddc012420 con 0x7efde4107940 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.869+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0115c0 con 0x7efde4107940 2024-04-25T13:03:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.870+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc8005330 con 0x7efde4107940 2024-04-25T13:03:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.872+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efddc019020 con 0x7efde4107940 2024-04-25T13:03:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.872+0000 7efddaffd700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7efdd00a3d50 0x7efdd00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.873+0000 7efde9a13700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7efdd00a3d50 0x7efdd00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.873+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7efddc0d0570 con 0x7efde4107940 2024-04-25T13:03:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.873+0000 7efde9a13700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7efdd00a3d50 0x7efdd00a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efde0005950 tx=0x7efde000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:47.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:47.877+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efddc020020 con 0x7efde4107940 2024-04-25T13:03:48.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.099+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efdc80058f0 con 0x7efde4107940 2024-04-25T13:03:48.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.100+0000 7efddaffd700 1 -- 172.21.15.38:0/3247304713 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efddc01a3b0 con 0x7efde4107940 2024-04-25T13:03:48.099 INFO:teuthology.orchestra.run.smithi038.stdout:201863462963 2024-04-25T13:03:48.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.104+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7efdd00a3d50 msgr2=0x7efdd00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.104+0000 7efdebc77700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7efdd00a3d50 0x7efdd00a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efde0005950 tx=0x7efde000b410 comp rx=0 tx=0).stop 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.104+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 msgr2=0x7efde419da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.104+0000 7efdebc77700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 0x7efde419da10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efddc00cca0 tx=0x7efddc00a060 comp rx=0 tx=0).stop 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.105+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 shutdown_connections 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.105+0000 7efdebc77700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7efdd00a3d50 0x7efdd00a6210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.105+0000 7efdebc77700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efde4102af0 0x7efde4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.105+0000 7efdebc77700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efde4107940 0x7efde419da10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.105+0000 7efdebc77700 1 --2- 172.21.15.38:0/3247304713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efde41034b0 0x7efde4199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.105+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 >> 172.21.15.38:0/3247304713 conn(0x7efde4079ba0 msgr2=0x7efde4100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.106+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 shutdown_connections 2024-04-25T13:03:48.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:48.106+0000 7efdebc77700 1 -- 172.21.15.38:0/3247304713 wait complete. 2024-04-25T13:03:48.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:48 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/750192667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:03:48.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:48 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3247304713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:03:48.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/750192667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:03:48.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3247304713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:03:48.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:48 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/750192667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:03:48.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:48 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3247304713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 -- 172.21.15.38:0/2789388384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de8101d20 msgr2=0x7f2de81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 --2- 172.21.15.38:0/2789388384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de8101d20 0x7f2de81021a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009990 tx=0x7f2dd8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 -- 172.21.15.38:0/2789388384 shutdown_connections 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 --2- 172.21.15.38:0/2789388384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2de8106990 0x7f2de8106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 --2- 172.21.15.38:0/2789388384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de81026e0 0x7f2de810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 --2- 172.21.15.38:0/2789388384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de8101d20 0x7f2de81021a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 -- 172.21.15.38:0/2789388384 >> 172.21.15.38:0/2789388384 conn(0x7f2de80788f0 msgr2=0x7f2de8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 -- 172.21.15.38:0/2789388384 shutdown_connections 2024-04-25T13:03:48.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.034+0000 7f2def971700 1 -- 172.21.15.38:0/2789388384 wait complete. 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 Processor -- start 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 -- start start 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8101d20 0x7f2de81035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 0x7f2de8103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2de8106990 0x7f2de8104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8111c60 con 0x7f2de81026e0 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8111ae0 con 0x7f2de8106990 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2def971700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8111de0 con 0x7f2de8101d20 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2decf0c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 0x7f2de8103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2decf0c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 0x7f2de8103b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51280/0 (socket says 172.21.15.38:51280) 2024-04-25T13:03:48.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2decf0c700 1 -- 172.21.15.38:0/2760210632 learned_addr learned my addr 172.21.15.38:0/2760210632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.035+0000 7f2dedf0e700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2de8106990 0x7f2de8104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2ded70d700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8101d20 0x7f2de81035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2decf0c700 1 -- 172.21.15.38:0/2760210632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8101d20 msgr2=0x7f2de81035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2decf0c700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8101d20 0x7f2de81035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2decf0c700 1 -- 172.21.15.38:0/2760210632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2de8106990 msgr2=0x7f2de8104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2decf0c700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2de8106990 0x7f2de8104060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2decf0c700 1 -- 172.21.15.38:0/2760210632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de81a7650 con 0x7f2de81026e0 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2decf0c700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 0x7f2de8103b20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8000c00 tx=0x7f2dd8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8026070 con 0x7f2de81026e0 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de81a78e0 con 0x7f2de81026e0 2024-04-25T13:03:48.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2dd8005500 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.036+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8003d40 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.037+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de81a7d30 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.038+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de8103080 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.039+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2dd8029070 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.039+0000 7f2dde7fc700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2dd40a3cd0 0x7f2dd40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.040+0000 7f2ded70d700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2dd40a3cd0 0x7f2dd40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.040+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2dd80d0ec0 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.040+0000 7f2ded70d700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2dd40a3cd0 0x7f2dd40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005950 tx=0x7f2de400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.045+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dd805dff0 con 0x7f2de81026e0 2024-04-25T13:03:48.866 INFO:teuthology.orchestra.run.smithi038.stdout:347892350999 2024-04-25T13:03:48.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.269+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2de802df80 con 0x7f2de81026e0 2024-04-25T13:03:48.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.269+0000 7f2dde7fc700 1 -- 172.21.15.38:0/2760210632 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2dd8013070 con 0x7f2de81026e0 2024-04-25T13:03:48.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.271+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2dd40a3cd0 msgr2=0x7f2dd40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.271+0000 7f2def971700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2dd40a3cd0 0x7f2dd40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005950 tx=0x7f2de400b410 comp rx=0 tx=0).stop 2024-04-25T13:03:48.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 msgr2=0x7f2de8103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 0x7f2de8103b20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8000c00 tx=0x7f2dd8009f50 comp rx=0 tx=0).stop 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 shutdown_connections 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f2dd40a3cd0 0x7f2dd40a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2de8106990 0x7f2de8104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8101d20 0x7f2de81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 --2- 172.21.15.38:0/2760210632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de81026e0 0x7f2de8103b20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 >> 172.21.15.38:0/2760210632 conn(0x7f2de80788f0 msgr2=0x7f2de806b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 shutdown_connections 2024-04-25T13:03:48.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:40.272+0000 7f2def971700 1 -- 172.21.15.38:0/2760210632 wait complete. 2024-04-25T13:03:49.424 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:49 smithi038 ceph-mon[29400]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 29 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:03:49.507 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:49 smithi178 ceph-mon[33233]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 29 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:03:49.508 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:49 smithi104 ceph-mon[32726]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 29 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T13:03:49.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350995 got 347892350999 for osd.11 2024-04-25T13:03:49.509 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:50.305 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299438 for osd.7 2024-04-25T13:03:50.305 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:50.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:50 smithi178 ceph-mon[33233]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 24 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:50.502 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:50 smithi038 ceph-mon[29400]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 24 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:50.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:50 smithi104 ceph-mon[32726]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 24 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.691+0000 7f3fe7470700 1 -- 172.21.15.38:0/2940426279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0102300 msgr2=0x7f3fe010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.691+0000 7f3fe7470700 1 --2- 172.21.15.38:0/2940426279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0102300 0x7f3fe010e7f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b0a0 tx=0x7f3fd800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 -- 172.21.15.38:0/2940426279 shutdown_connections 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 --2- 172.21.15.38:0/2940426279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0102300 0x7f3fe010e7f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 --2- 172.21.15.38:0/2940426279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0107990 0x7f3fe0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 --2- 172.21.15.38:0/2940426279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0101940 0x7f3fe0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 -- 172.21.15.38:0/2940426279 >> 172.21.15.38:0/2940426279 conn(0x7f3fe00788f0 msgr2=0x7f3fe0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 -- 172.21.15.38:0/2940426279 shutdown_connections 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 -- 172.21.15.38:0/2940426279 wait complete. 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 Processor -- start 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 -- start start 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.692+0000 7f3fe7470700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 0x7f3fe019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0102300 0x7f3fe019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0107990 0x7f3fe019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0111c70 con 0x7f3fe0101940 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0111af0 con 0x7f3fe0107990 2024-04-25T13:03:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0111df0 con 0x7f3fe0102300 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 0x7f3fe019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 0x7f3fe019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38116/0 (socket says 172.21.15.38:38116) 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 -- 172.21.15.38:0/4009939953 learned_addr learned my addr 172.21.15.38:0/4009939953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe4a0b700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0102300 0x7f3fe019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe5a0d700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0107990 0x7f3fe019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 -- 172.21.15.38:0/4009939953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0102300 msgr2=0x7f3fe019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0102300 0x7f3fe019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 -- 172.21.15.38:0/4009939953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0107990 msgr2=0x7f3fe019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0107990 0x7f3fe019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 -- 172.21.15.38:0/4009939953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a0340 con 0x7f3fe0101940 2024-04-25T13:03:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe4a0b700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0102300 0x7f3fe019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe5a0d700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0107990 0x7f3fe019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe520c700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 0x7f3fe019b330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9a0 tx=0x7f3fdc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00cbd0 con 0x7f3fe0101940 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a0630 con 0x7f3fe0101940 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3fdc00cd70 con 0x7f3fe0101940 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0187a0 con 0x7f3fe0101940 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.693+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a7eb0 con 0x7f3fe0101940 2024-04-25T13:03:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.694+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe0069730 con 0x7f3fe0101940 2024-04-25T13:03:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.696+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3fdc010a20 con 0x7f3fe0101940 2024-04-25T13:03:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.696+0000 7f3fd67fc700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.696+0000 7f3fe4a0b700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.696+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3fdc0d0a40 con 0x7f3fe0101940 2024-04-25T13:03:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.697+0000 7f3fe4a0b700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3fe01018a0 tx=0x7f3fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.700+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fdc09d2c0 con 0x7f3fe0101940 2024-04-25T13:03:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:03:50.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.930+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3fe019c690 con 0x7f3fe0101940 2024-04-25T13:03:50.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.930+0000 7f3fd67fc700 1 -- 172.21.15.38:0/4009939953 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3fdc09cbc0 con 0x7f3fe0101940 2024-04-25T13:03:50.930 INFO:teuthology.orchestra.run.smithi038.stdout:111669149763 2024-04-25T13:03:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.932+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3fcc0a3cd0 msgr2=0x7f3fcc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.932+0000 7f3fe7470700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3fe01018a0 tx=0x7f3fd0009260 comp rx=0 tx=0).stop 2024-04-25T13:03:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.932+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 msgr2=0x7f3fe019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 0x7f3fe019b330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9a0 tx=0x7f3fdc00ed60 comp rx=0 tx=0).stop 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 shutdown_connections 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0107990 0x7f3fe019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fe0101940 0x7f3fe019b330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 --2- 172.21.15.38:0/4009939953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe0102300 0x7f3fe019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 >> 172.21.15.38:0/4009939953 conn(0x7f3fe00788f0 msgr2=0x7f3fe006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 shutdown_connections 2024-04-25T13:03:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.933+0000 7f3fe7470700 1 -- 172.21.15.38:0/4009939953 wait complete. 2024-04-25T13:03:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 -- 172.21.15.38:0/2074451637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464101940 msgr2=0x7ff464101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 --2- 172.21.15.38:0/2074451637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464101940 0x7ff464101dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff454009990 tx=0x7ff454009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 -- 172.21.15.38:0/2074451637 shutdown_connections 2024-04-25T13:03:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 --2- 172.21.15.38:0/2074451637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff464102300 0x7ff46410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 --2- 172.21.15.38:0/2074451637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464107990 0x7ff464107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 --2- 172.21.15.38:0/2074451637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464101940 0x7ff464101dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 -- 172.21.15.38:0/2074451637 >> 172.21.15.38:0/2074451637 conn(0x7ff4640788f0 msgr2=0x7ff464078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 -- 172.21.15.38:0/2074451637 shutdown_connections 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.939+0000 7ff46b31f700 1 -- 172.21.15.38:0/2074451637 wait complete. 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 Processor -- start 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 -- start start 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464101940 0x7ff464198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 0x7ff464198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff464107990 0x7ff46419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff464111c70 con 0x7ff464102300 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff464111af0 con 0x7ff464107990 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff46b31f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff464111df0 con 0x7ff464101940 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff4688ba700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 0x7ff464198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff4688ba700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 0x7ff464198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38138/0 (socket says 172.21.15.38:38138) 2024-04-25T13:03:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.940+0000 7ff4688ba700 1 -- 172.21.15.38:0/90599048 learned_addr learned my addr 172.21.15.38:0/90599048 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4698bc700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff464107990 0x7ff46419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4690bb700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464101940 0x7ff464198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4688ba700 1 -- 172.21.15.38:0/90599048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464101940 msgr2=0x7ff464198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4688ba700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464101940 0x7ff464198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4688ba700 1 -- 172.21.15.38:0/90599048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff464107990 msgr2=0x7ff46419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4688ba700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff464107990 0x7ff46419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4688ba700 1 -- 172.21.15.38:0/90599048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419d0c0 con 0x7ff464102300 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4690bb700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464101940 0x7ff464198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff4688ba700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 0x7ff464198640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454029070 con 0x7ff464102300 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46419d350 con 0x7ff464102300 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff454019ce0 con 0x7ff464102300 2024-04-25T13:03:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454024470 con 0x7ff464102300 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.941+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46406a720 con 0x7ff464102300 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.942+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff464069730 con 0x7ff464102300 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.944+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff45402c070 con 0x7ff464102300 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.945+0000 7ff45a7fc700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff4500a3cd0 0x7ff4500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.945+0000 7ff4690bb700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff4500a3cd0 0x7ff4500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.945+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff4540d3730 con 0x7ff464102300 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.945+0000 7ff4690bb700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff4500a3cd0 0x7ff4500a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff460009b50 tx=0x7ff460009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:50.949+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff454033080 con 0x7ff464102300 2024-04-25T13:03:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.176+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff464199410 con 0x7ff464102300 2024-04-25T13:03:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.177+0000 7ff45a7fc700 1 -- 172.21.15.38:0/90599048 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff464199410 con 0x7ff464102300 2024-04-25T13:03:51.177 INFO:teuthology.orchestra.run.smithi038.stdout:55834574926 2024-04-25T13:03:51.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff4500a3cd0 msgr2=0x7ff4500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff4500a3cd0 0x7ff4500a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff460009b50 tx=0x7ff460009550 comp rx=0 tx=0).stop 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 msgr2=0x7ff464198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 0x7ff464198640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454004580 comp rx=0 tx=0).stop 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 shutdown_connections 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff4500a3cd0 0x7ff4500a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff464107990 0x7ff46419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff464101940 0x7ff464198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 --2- 172.21.15.38:0/90599048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff464102300 0x7ff464198640 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 >> 172.21.15.38:0/90599048 conn(0x7ff4640788f0 msgr2=0x7ff4641006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 shutdown_connections 2024-04-25T13:03:51.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:51.179+0000 7ff46b31f700 1 -- 172.21.15.38:0/90599048 wait complete. 2024-04-25T13:03:51.426 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:51 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/4009939953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:03:51.426 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:51 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/90599048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:03:51.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:51 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/4009939953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:03:51.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:51 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/90599048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:03:51.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:51 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/4009939953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:03:51.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:51 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/90599048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:03:51.858 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149763 for osd.3 2024-04-25T13:03:51.858 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2905605806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc102ac0 msgr2=0x7fd7cc102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2905605806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc102ac0 0x7fd7cc102f40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009990 tx=0x7fd7c0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2905605806 shutdown_connections 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2905605806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc103480 0x7fd7cc10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2905605806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7cc108a70 0x7fd7cc108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2905605806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc102ac0 0x7fd7cc102f40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2905605806 >> 172.21.15.38:0/2905605806 conn(0x7fd7cc079bb0 msgr2=0x7fd7cc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2905605806 shutdown_connections 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.258+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2905605806 wait complete. 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 Processor -- start 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 -- start start 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7cc102ac0 0x7fd7cc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:52.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 0x7fd7cc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc108a70 0x7fd7cc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc112de0 con 0x7fd7cc103480 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc112c60 con 0x7fd7cc108a70 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7d15a5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc112f60 con 0x7fd7cc102ac0 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7ca7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 0x7fd7cc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7ca7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 0x7fd7cc1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38152/0 (socket says 172.21.15.38:38152) 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7ca7fc700 1 -- 172.21.15.38:0/2415283578 learned_addr learned my addr 172.21.15.38:0/2415283578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7caffd700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7cc102ac0 0x7fd7cc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.259+0000 7fd7cb7fe700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc108a70 0x7fd7cc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7ca7fc700 1 -- 172.21.15.38:0/2415283578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7cc102ac0 msgr2=0x7fd7cc1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7ca7fc700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7cc102ac0 0x7fd7cc1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7ca7fc700 1 -- 172.21.15.38:0/2415283578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc108a70 msgr2=0x7fd7cc19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7ca7fc700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc108a70 0x7fd7cc19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7ca7fc700 1 -- 172.21.15.38:0/2415283578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc19e2c0 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7cb7fe700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc108a70 0x7fd7cc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7ca7fc700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 0x7fd7cc1997f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000c00 tx=0x7fd7c0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0027070 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd7c0003fb0 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc19e550 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0022e10 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.260+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc1a5e30 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.261+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7cc1041a0 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.262+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd7c002a070 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.263+0000 7fd7b3fff700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd7b40a3cd0 0x7fd7b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.263+0000 7fd7caffd700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd7b40a3cd0 0x7fd7b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.263+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd7c00d1310 con 0x7fd7cc103480 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.263+0000 7fd7caffd700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd7b40a3cd0 0x7fd7b40a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc004630 tx=0x7fd7bc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.267+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7c005eff0 con 0x7fd7cc103480 2024-04-25T13:03:52.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:52 smithi038 ceph-mon[29400]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 24 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:52.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.485+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd7cc02df80 con 0x7fd7cc103480 2024-04-25T13:03:52.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.485+0000 7fd7b3fff700 1 -- 172.21.15.38:0/2415283578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd7c0013070 con 0x7fd7cc103480 2024-04-25T13:03:52.485 INFO:teuthology.orchestra.run.smithi038.stdout:146028888127 2024-04-25T13:03:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd7b40a3cd0 msgr2=0x7fd7b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd7b40a3cd0 0x7fd7b40a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc004630 tx=0x7fd7bc00b3c0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 msgr2=0x7fd7cc1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 0x7fd7cc1997f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000c00 tx=0x7fd7c0009f50 comp rx=0 tx=0).stop 2024-04-25T13:03:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 shutdown_connections 2024-04-25T13:03:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fd7b40a3cd0 0x7fd7b40a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7cc108a70 0x7fd7cc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7cc102ac0 0x7fd7cc1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 --2- 172.21.15.38:0/2415283578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7cc103480 0x7fd7cc1997f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:52.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 >> 172.21.15.38:0/2415283578 conn(0x7fd7cc079bb0 msgr2=0x7fd7cc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:52.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 shutdown_connections 2024-04-25T13:03:52.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:52.488+0000 7fd7d15a5700 1 -- 172.21.15.38:0/2415283578 wait complete. 2024-04-25T13:03:52.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:52 smithi178 ceph-mon[33233]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 24 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:52.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:52 smithi104 ceph-mon[32726]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 24 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:53.035 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462963 for osd.6 2024-04-25T13:03:53.035 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:53.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:53 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2415283578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:03:53.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:53 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2415283578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:03:53.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:53 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2415283578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:03:54.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:54 smithi038 ceph-mon[29400]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:54.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:54 smithi178 ceph-mon[33233]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:54.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:54 smithi104 ceph-mon[32726]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:03:54.778 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888121 got 146028888127 for osd.4 2024-04-25T13:03:54.778 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:55.643 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574926 for osd.1 2024-04-25T13:03:55.643 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:56.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:56 smithi178 ceph-mon[33233]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:03:56.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:56 smithi038 ceph-mon[29400]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:03:56.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:56 smithi104 ceph-mon[32726]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:03:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 -- 172.21.15.38:0/2734974755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006b170 msgr2=0x7fe30006b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 --2- 172.21.15.38:0/2734974755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006b170 0x7fe30006b5d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009990 tx=0x7fe2f4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 -- 172.21.15.38:0/2734974755 shutdown_connections 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 --2- 172.21.15.38:0/2734974755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe30006bb10 0x7fe30010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 --2- 172.21.15.38:0/2734974755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe300108a30 0x7fe300108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 --2- 172.21.15.38:0/2734974755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006b170 0x7fe30006b5d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 -- 172.21.15.38:0/2734974755 >> 172.21.15.38:0/2734974755 conn(0x7fe3000788f0 msgr2=0x7fe300078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 -- 172.21.15.38:0/2734974755 shutdown_connections 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.833+0000 7fe3054f1700 1 -- 172.21.15.38:0/2734974755 wait complete. 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.834+0000 7fe3054f1700 1 Processor -- start 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.834+0000 7fe3054f1700 1 -- start start 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.834+0000 7fe3054f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe30006b170 0x7fe300199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.834+0000 7fe3054f1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 0x7fe3001997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.834+0000 7fe3054f1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe300108a30 0x7fe30019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe3054f1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe300112d90 con 0x7fe30006bb10 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe3054f1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe300112c10 con 0x7fe300108a30 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe3054f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe300112f10 con 0x7fe30006b170 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 0x7fe3001997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 0x7fe3001997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38178/0 (socket says 172.21.15.38:38178) 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 -- 172.21.15.38:0/1985640209 learned_addr learned my addr 172.21.15.38:0/1985640209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2ff7fe700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe300108a30 0x7fe30019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2feffd700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe30006b170 0x7fe300199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 -- 172.21.15.38:0/1985640209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe30006b170 msgr2=0x7fe300199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe30006b170 0x7fe300199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 -- 172.21.15.38:0/1985640209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe300108a30 msgr2=0x7fe30019db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe300108a30 0x7fe30019db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 -- 172.21.15.38:0/1985640209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30019e270 con 0x7fe30006bb10 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2feffd700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe30006b170 0x7fe300199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:03:57.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2fe7fc700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 0x7fe3001997a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4000c00 tx=0x7fe2f4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4027070 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30019e500 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe2f4003fb0 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4022e60 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.835+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3001a5de0 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.836+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe300069730 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.837+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe2f402a070 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.838+0000 7fe2e7fff700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe2e80a3cd0 0x7fe2e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.838+0000 7fe2feffd700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe2e80a3cd0 0x7fe2e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.838+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe2f40d1290 con 0x7fe30006bb10 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.838+0000 7fe2feffd700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe2e80a3cd0 0x7fe2e80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0007900 tx=0x7fe2f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:57.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:57.843+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2f4013070 con 0x7fe30006bb10 2024-04-25T13:03:58.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.064+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe30002df80 con 0x7fe30006bb10 2024-04-25T13:03:58.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.064+0000 7fe2e7fff700 1 -- 172.21.15.38:0/1985640209 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe2f409db60 con 0x7fe30006bb10 2024-04-25T13:03:58.064 INFO:teuthology.orchestra.run.smithi038.stdout:167503724603 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe2e80a3cd0 msgr2=0x7fe2e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe2e80a3cd0 0x7fe2e80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0007900 tx=0x7fe2f0008040 comp rx=0 tx=0).stop 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 msgr2=0x7fe3001997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 0x7fe3001997a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4000c00 tx=0x7fe2f4004580 comp rx=0 tx=0).stop 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 shutdown_connections 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe2e80a3cd0 0x7fe2e80a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe300108a30 0x7fe30019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe30006b170 0x7fe300199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 --2- 172.21.15.38:0/1985640209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe30006bb10 0x7fe3001997a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.066+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 >> 172.21.15.38:0/1985640209 conn(0x7fe3000788f0 msgr2=0x7fe3000ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.067+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 shutdown_connections 2024-04-25T13:03:58.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.067+0000 7fe3054f1700 1 -- 172.21.15.38:0/1985640209 wait complete. 2024-04-25T13:03:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 -- 172.21.15.38:0/1946351653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c06b0f0 msgr2=0x7f111c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 --2- 172.21.15.38:0/1946351653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c06b0f0 0x7f111c10f900 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f111400b0a0 tx=0x7f111400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 -- 172.21.15.38:0/1946351653 shutdown_connections 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 --2- 172.21.15.38:0/1946351653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c108a30 0x7f111c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 --2- 172.21.15.38:0/1946351653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c06b0f0 0x7f111c10f900 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 --2- 172.21.15.38:0/1946351653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f111c06a750 0x7f111c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 -- 172.21.15.38:0/1946351653 >> 172.21.15.38:0/1946351653 conn(0x7f111c0788f0 msgr2=0x7f111c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 -- 172.21.15.38:0/1946351653 shutdown_connections 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.517+0000 7f11211d8700 1 -- 172.21.15.38:0/1946351653 wait complete. 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.518+0000 7f11211d8700 1 Processor -- start 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 -- start start 2024-04-25T13:03:58.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c06a750 0x7f111c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f111c06b0f0 0x7f111c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 0x7f111c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f111c112d90 con 0x7f111c108a30 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f111c112c10 con 0x7f111c06a750 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f11211d8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f111c112f10 con 0x7f111c06b0f0 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 0x7f111c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 0x7f111c102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38198/0 (socket says 172.21.15.38:38198) 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 -- 172.21.15.38:0/625832583 learned_addr learned my addr 172.21.15.38:0/625832583 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111a59c700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f111c06b0f0 0x7f111c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111ad9d700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c06a750 0x7f111c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 -- 172.21.15.38:0/625832583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f111c06b0f0 msgr2=0x7f111c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f111c06b0f0 0x7f111c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 -- 172.21.15.38:0/625832583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c06a750 msgr2=0x7f111c101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c06a750 0x7f111c101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.519+0000 7f111b59e700 1 -- 172.21.15.38:0/625832583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c1a76b0 con 0x7f111c108a30 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.520+0000 7f111b59e700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 0x7f111c102490 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f111400b070 tx=0x7f1114007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.520+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114021070 con 0x7f111c108a30 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.520+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1114007e50 con 0x7f111c108a30 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.520+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c1a7940 con 0x7f111c108a30 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.520+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111401ba90 con 0x7f111c108a30 2024-04-25T13:03:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.520+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1a7df0 con 0x7f111c108a30 2024-04-25T13:03:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.521+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c069730 con 0x7f111c108a30 2024-04-25T13:03:58.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.522+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1114024070 con 0x7f111c108a30 2024-04-25T13:03:58.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.523+0000 7f1103fff700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f11040a3cd0 0x7f11040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:58.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.523+0000 7f111ad9d700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f11040a3cd0 0x7f11040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:58.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.523+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f111400f020 con 0x7f111c108a30 2024-04-25T13:03:58.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.524+0000 7f111ad9d700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f11040a3cd0 0x7f11040a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f110c005950 tx=0x7f110c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.527+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f111402b020 con 0x7f111c108a30 2024-04-25T13:03:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:58 smithi038 ceph-mon[29400]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:58 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1985640209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:03:58.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:58 smithi178 ceph-mon[33233]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:58.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:58 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1985640209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:03:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.743+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f111c02e140 con 0x7f111c108a30 2024-04-25T13:03:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.743+0000 7f1103fff700 1 -- 172.21.15.38:0/625832583 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f111409ee20 con 0x7f111c108a30 2024-04-25T13:03:58.743 INFO:teuthology.orchestra.run.smithi038.stdout:257698037803 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f11040a3cd0 msgr2=0x7f11040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f11040a3cd0 0x7f11040a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f110c005950 tx=0x7f110c00b410 comp rx=0 tx=0).stop 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 msgr2=0x7f111c102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 0x7f111c102490 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f111400b070 tx=0x7f1114007a20 comp rx=0 tx=0).stop 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 shutdown_connections 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f11040a3cd0 0x7f11040a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c06a750 0x7f111c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f111c108a30 0x7f111c102490 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 --2- 172.21.15.38:0/625832583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f111c06b0f0 0x7f111c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.745+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 >> 172.21.15.38:0/625832583 conn(0x7f111c0788f0 msgr2=0x7f111c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.746+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 shutdown_connections 2024-04-25T13:03:58.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:58.746+0000 7f11211d8700 1 -- 172.21.15.38:0/625832583 wait complete. 2024-04-25T13:03:58.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:58 smithi104 ceph-mon[32726]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:03:58.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:58 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1985640209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:03:59.043 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724597 got 167503724603 for osd.5 2024-04-25T13:03:59.043 DEBUG:teuthology.parallel:result is None 2024-04-25T13:03:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 -- 172.21.15.38:0/3825437102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 msgr2=0x7f1790101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 --2- 172.21.15.38:0/3825437102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f1790101dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1784009990 tx=0x7f1784009ca0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 -- 172.21.15.38:0/3825437102 shutdown_connections 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 --2- 172.21.15.38:0/3825437102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1790102300 0x7f179010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 --2- 172.21.15.38:0/3825437102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790107990 0x7f1790107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 --2- 172.21.15.38:0/3825437102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f1790101dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.236+0000 7f1795f72700 1 -- 172.21.15.38:0/3825437102 >> 172.21.15.38:0/3825437102 conn(0x7f17900788f0 msgr2=0x7f1790078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.237+0000 7f1795f72700 1 -- 172.21.15.38:0/3825437102 shutdown_connections 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.237+0000 7f1795f72700 1 -- 172.21.15.38:0/3825437102 wait complete. 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.237+0000 7f1795f72700 1 Processor -- start 2024-04-25T13:03:59.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.237+0000 7f1795f72700 1 -- start start 2024-04-25T13:03:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f1795f72700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f179019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f1795f72700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790102300 0x7f179019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f1795f72700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1790107990 0x7f17901a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f1795f72700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111c70 con 0x7f1790101940 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f1795f72700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111af0 con 0x7f1790107990 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f1795f72700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111df0 con 0x7f1790102300 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f179019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f179019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38212/0 (socket says 172.21.15.38:38212) 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 -- 172.21.15.38:0/3038838993 learned_addr learned my addr 172.21.15.38:0/3038838993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178effd700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790102300 0x7f179019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178ffff700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1790107990 0x7f17901a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 -- 172.21.15.38:0/3038838993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790102300 msgr2=0x7f179019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790102300 0x7f179019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 -- 172.21.15.38:0/3038838993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1790107990 msgr2=0x7f17901a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1790107990 0x7f17901a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 -- 172.21.15.38:0/3038838993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a55f0 con 0x7f1790101940 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178f7fe700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f179019a030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17800060b0 tx=0x7f178000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.238+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178001a070 con 0x7f1790101940 2024-04-25T13:03:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.239+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a5820 con 0x7f1790101940 2024-04-25T13:03:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.239+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f178000ddd0 con 0x7f1790101940 2024-04-25T13:03:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.239+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800147c0 con 0x7f1790101940 2024-04-25T13:03:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.239+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a5d80 con 0x7f1790101940 2024-04-25T13:03:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.240+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c005330 con 0x7f1790101940 2024-04-25T13:03:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.241+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f178001d070 con 0x7f1790101940 2024-04-25T13:03:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.241+0000 7f178cff9700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f17780a3d50 0x7f17780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:03:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.241+0000 7f178effd700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f17780a3d50 0x7f17780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:03:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.241+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f17800d0010 con 0x7f1790101940 2024-04-25T13:03:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.242+0000 7f178effd700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f17780a3d50 0x7f17780a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f178400b5c0 tx=0x7f1784005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:03:59.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.245+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1780010070 con 0x7f1790101940 2024-04-25T13:03:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.459+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f177c0058f0 con 0x7f1790101940 2024-04-25T13:03:59.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.460+0000 7f178cff9700 1 -- 172.21.15.38:0/3038838993 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f178009c890 con 0x7f1790101940 2024-04-25T13:03:59.459 INFO:teuthology.orchestra.run.smithi038.stdout:77309411402 2024-04-25T13:03:59.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f17780a3d50 msgr2=0x7f17780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f17780a3d50 0x7f17780a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f178400b5c0 tx=0x7f1784005960 comp rx=0 tx=0).stop 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 msgr2=0x7f179019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f179019a030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17800060b0 tx=0x7f178000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 shutdown_connections 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f17780a3d50 0x7f17780a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1790107990 0x7f17901a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790101940 0x7f179019a030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 --2- 172.21.15.38:0/3038838993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790102300 0x7f179019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 >> 172.21.15.38:0/3038838993 conn(0x7f17900788f0 msgr2=0x7f17901006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 shutdown_connections 2024-04-25T13:03:59.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:03:59.462+0000 7f1795f72700 1 -- 172.21.15.38:0/3038838993 wait complete. 2024-04-25T13:03:59.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:03:59 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/625832583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:03:59.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:03:59 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/625832583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:03:59.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:03:59 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/625832583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:04:00.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037803 for osd.8 2024-04-25T13:04:00.160 DEBUG:teuthology.parallel:result is None 2024-04-25T13:04:00.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:00 smithi038 ceph-mon[29400]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:00.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:00 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3038838993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:04:00.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:00 smithi178 ceph-mon[33233]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:00.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:00 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3038838993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:04:00.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:00 smithi104 ceph-mon[32726]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:00.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:00 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3038838993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:04:00.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411402 for osd.2 2024-04-25T13:04:00.965 DEBUG:teuthology.parallel:result is None 2024-04-25T13:04:00.965 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:04:00.965 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:04:01.248 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:02.664 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:02 smithi038 ceph-mon[29400]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:04:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:02 smithi178 ceph-mon[33233]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:04:02.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:02 smithi104 ceph-mon[32726]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 -- 172.21.15.38:0/2468318716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104806b170 msgr2=0x7f104806b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 --2- 172.21.15.38:0/2468318716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104806b170 0x7f104806b5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f103c009990 tx=0x7f103c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 -- 172.21.15.38:0/2468318716 shutdown_connections 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 --2- 172.21.15.38:0/2468318716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 0x7f104810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 --2- 172.21.15.38:0/2468318716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1048108a30 0x7f1048108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 --2- 172.21.15.38:0/2468318716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104806b170 0x7f104806b5d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 -- 172.21.15.38:0/2468318716 >> 172.21.15.38:0/2468318716 conn(0x7f10480788f0 msgr2=0x7f1048078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 -- 172.21.15.38:0/2468318716 shutdown_connections 2024-04-25T13:04:02.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 -- 172.21.15.38:0/2468318716 wait complete. 2024-04-25T13:04:02.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.873+0000 7f104d5f3700 1 Processor -- start 2024-04-25T13:04:02.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 -- start start 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f104806b170 0x7f1048101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 0x7f1048101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 0x7f10481024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1048112de0 con 0x7f1048108a30 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1048112c60 con 0x7f104806bb10 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f104d5f3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1048112f60 con 0x7f104806b170 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 0x7f10481024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f104806b170 0x7f1048101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 0x7f10481024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58768/0 (socket says 172.21.15.38:58768) 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 -- 172.21.15.38:0/3041684574 learned_addr learned my addr 172.21.15.38:0/3041684574 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f104806b170 0x7f1048101a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.38:42668/0 (socket says 172.21.15.38:42668) 2024-04-25T13:04:02.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10467fc700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 0x7f1048101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 -- 172.21.15.38:0/3041684574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f104806b170 msgr2=0x7f1048101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f104806b170 0x7f1048101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 -- 172.21.15.38:0/3041684574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 msgr2=0x7f1048101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 0x7f1048101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.874+0000 7f10477fe700 1 -- 172.21.15.38:0/3041684574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10481a7720 con 0x7f1048108a30 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f10467fc700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 0x7f1048101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f10477fe700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 0x7f10481024c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1040007d70 tx=0x7f104000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040016020 con 0x7f1048108a30 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f104000ad40 con 0x7f1048108a30 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10481a7a10 con 0x7f1048108a30 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400116b0 con 0x7f1048108a30 2024-04-25T13:04:02.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.875+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481a7f00 con 0x7f1048108a30 2024-04-25T13:04:02.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.876+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1048069730 con 0x7f1048108a30 2024-04-25T13:04:02.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.877+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1040019020 con 0x7f1048108a30 2024-04-25T13:04:02.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.878+0000 7f102ffff700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f10300a3cd0 0x7f10300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:02.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.878+0000 7f1046ffd700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f10300a3cd0 0x7f10300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:02.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.878+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f10400d0450 con 0x7f1048108a30 2024-04-25T13:04:02.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.878+0000 7f1046ffd700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f10300a3cd0 0x7f10300a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1038005950 tx=0x7f103800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:02.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:02.883+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1040020020 con 0x7f1048108a30 2024-04-25T13:04:03.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.095+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1048062e80 con 0x7f10300a3cd0 2024-04-25T13:04:03.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.102+0000 7f102ffff700 1 -- 172.21.15.38:0/3041684574 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259078 (secure 0 0 0) 0x7f1048062e80 con 0x7f10300a3cd0 2024-04-25T13:04:03.106 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:03.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f10300a3cd0 msgr2=0x7f10300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:03.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f10300a3cd0 0x7f10300a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1038005950 tx=0x7f103800a300 comp rx=0 tx=0).stop 2024-04-25T13:04:03.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 msgr2=0x7f10481024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:03.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 0x7f10481024c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1040007d70 tx=0x7f104000a340 comp rx=0 tx=0).stop 2024-04-25T13:04:03.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 shutdown_connections 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f10300a3cd0 0x7f10300a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1048108a30 0x7f10481024c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f104806b170 0x7f1048101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 --2- 172.21.15.38:0/3041684574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f104806bb10 0x7f1048101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.105+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 >> 172.21.15.38:0/3041684574 conn(0x7f10480788f0 msgr2=0x7f10480ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.106+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 shutdown_connections 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:03.106+0000 7f104d5f3700 1 -- 172.21.15.38:0/3041684574 wait complete. 2024-04-25T13:04:03.109 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T13:04:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:03 smithi178 ceph-mon[33233]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:03 smithi178 ceph-mon[33233]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:03.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:03.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:03.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:03 smithi038 ceph-mon[29400]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:03 smithi038 ceph-mon[29400]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:03.776 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-04-25T13:04:01.301859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655936,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":857,"num_read_kb":552,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181033,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373496,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568328,"statfs":{"total":1151940427776,"available":1151557967872,"internally_reserved":0,"allocated":12079104,"data_stored":7473036,"data_compressed":185424,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003291"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120552+0000","last_change":"2024-04-25T13:02:44.736489+0000","last_active":"2024-04-25T13:03:37.120552+0000","last_peered":"2024-04-25T13:03:37.120552+0000","last_clean":"2024-04-25T13:03:37.120552+0000","last_became_active":"2024-04-25T13:02:44.735677+0000","last_became_peered":"2024-04-25T13:02:44.735677+0000","last_unstale":"2024-04-25T13:03:37.120552+0000","last_undegraded":"2024-04-25T13:03:37.120552+0000","last_fullsized":"2024-04-25T13:03:37.120552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:59.810878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.125057+0000","last_change":"2024-04-25T13:02:44.740349+0000","last_active":"2024-04-25T13:03:37.125057+0000","last_peered":"2024-04-25T13:03:37.125057+0000","last_clean":"2024-04-25T13:03:37.125057+0000","last_became_active":"2024-04-25T13:02:44.740024+0000","last_became_peered":"2024-04-25T13:02:44.740024+0000","last_unstale":"2024-04-25T13:03:37.125057+0000","last_undegraded":"2024-04-25T13:03:37.125057+0000","last_fullsized":"2024-04-25T13:03:37.125057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:35.763278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124578+0000","last_change":"2024-04-25T13:02:44.742136+0000","last_active":"2024-04-25T13:03:37.124578+0000","last_peered":"2024-04-25T13:03:37.124578+0000","last_clean":"2024-04-25T13:03:37.124578+0000","last_became_active":"2024-04-25T13:02:44.741840+0000","last_became_peered":"2024-04-25T13:02:44.741840+0000","last_unstale":"2024-04-25T13:03:37.124578+0000","last_undegraded":"2024-04-25T13:03:37.124578+0000","last_fullsized":"2024-04-25T13:03:37.124578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:48.690435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.210095+0000","last_change":"2024-04-25T13:02:44.733349+0000","last_active":"2024-04-25T13:03:37.210095+0000","last_peered":"2024-04-25T13:03:37.210095+0000","last_clean":"2024-04-25T13:03:37.210095+0000","last_became_active":"2024-04-25T13:02:44.732389+0000","last_became_peered":"2024-04-25T13:02:44.732389+0000","last_unstale":"2024-04-25T13:03:37.210095+0000","last_undegraded":"2024-04-25T13:03:37.210095+0000","last_fullsized":"2024-04-25T13:03:37.210095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:28.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124148+0000","last_change":"2024-04-25T13:02:44.732852+0000","last_active":"2024-04-25T13:03:37.124148+0000","last_peered":"2024-04-25T13:03:37.124148+0000","last_clean":"2024-04-25T13:03:37.124148+0000","last_became_active":"2024-04-25T13:02:44.732536+0000","last_became_peered":"2024-04-25T13:02:44.732536+0000","last_unstale":"2024-04-25T13:03:37.124148+0000","last_undegraded":"2024-04-25T13:03:37.124148+0000","last_fullsized":"2024-04-25T13:03:37.124148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:58.358684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.122892+0000","last_change":"2024-04-25T13:02:44.731636+0000","last_active":"2024-04-25T13:03:37.122892+0000","last_peered":"2024-04-25T13:03:37.122892+0000","last_clean":"2024-04-25T13:03:37.122892+0000","last_became_active":"2024-04-25T13:02:44.731334+0000","last_became_peered":"2024-04-25T13:02:44.731334+0000","last_unstale":"2024-04-25T13:03:37.122892+0000","last_undegraded":"2024-04-25T13:03:37.122892+0000","last_fullsized":"2024-04-25T13:03:37.122892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:00.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123336+0000","last_change":"2024-04-25T13:02:44.737275+0000","last_active":"2024-04-25T13:03:37.123336+0000","last_peered":"2024-04-25T13:03:37.123336+0000","last_clean":"2024-04-25T13:03:37.123336+0000","last_became_active":"2024-04-25T13:02:44.736924+0000","last_became_peered":"2024-04-25T13:02:44.736924+0000","last_unstale":"2024-04-25T13:03:37.123336+0000","last_undegraded":"2024-04-25T13:03:37.123336+0000","last_fullsized":"2024-04-25T13:03:37.123336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.564465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123131+0000","last_change":"2024-04-25T13:02:44.736736+0000","last_active":"2024-04-25T13:03:37.123131+0000","last_peered":"2024-04-25T13:03:37.123131+0000","last_clean":"2024-04-25T13:03:37.123131+0000","last_became_active":"2024-04-25T13:02:44.736403+0000","last_became_peered":"2024-04-25T13:02:44.736403+0000","last_unstale":"2024-04-25T13:03:37.123131+0000","last_undegraded":"2024-04-25T13:03:37.123131+0000","last_fullsized":"2024-04-25T13:03:37.123131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:54.531475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212262+0000","last_change":"2024-04-25T13:02:44.738124+0000","last_active":"2024-04-25T13:03:37.212262+0000","last_peered":"2024-04-25T13:03:37.212262+0000","last_clean":"2024-04-25T13:03:37.212262+0000","last_became_active":"2024-04-25T13:02:44.737572+0000","last_became_peered":"2024-04-25T13:02:44.737572+0000","last_unstale":"2024-04-25T13:03:37.212262+0000","last_undegraded":"2024-04-25T13:03:37.212262+0000","last_fullsized":"2024-04-25T13:03:37.212262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:00.692786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123141+0000","last_change":"2024-04-25T13:02:44.730108+0000","last_active":"2024-04-25T13:03:37.123141+0000","last_peered":"2024-04-25T13:03:37.123141+0000","last_clean":"2024-04-25T13:03:37.123141+0000","last_became_active":"2024-04-25T13:02:44.729449+0000","last_became_peered":"2024-04-25T13:02:44.729449+0000","last_unstale":"2024-04-25T13:03:37.123141+0000","last_undegraded":"2024-04-25T13:03:37.123141+0000","last_fullsized":"2024-04-25T13:03:37.123141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:19.888190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123345+0000","last_change":"2024-04-25T13:02:44.730902+0000","last_active":"2024-04-25T13:03:37.123345+0000","last_peered":"2024-04-25T13:03:37.123345+0000","last_clean":"2024-04-25T13:03:37.123345+0000","last_became_active":"2024-04-25T13:02:44.730560+0000","last_became_peered":"2024-04-25T13:02:44.730560+0000","last_unstale":"2024-04-25T13:03:37.123345+0000","last_undegraded":"2024-04-25T13:03:37.123345+0000","last_fullsized":"2024-04-25T13:03:37.123345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:07.133221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.210084+0000","last_change":"2024-04-25T13:02:44.730966+0000","last_active":"2024-04-25T13:03:37.210084+0000","last_peered":"2024-04-25T13:03:37.210084+0000","last_clean":"2024-04-25T13:03:37.210084+0000","last_became_active":"2024-04-25T13:02:44.730546+0000","last_became_peered":"2024-04-25T13:02:44.730546+0000","last_unstale":"2024-04-25T13:03:37.210084+0000","last_undegraded":"2024-04-25T13:03:37.210084+0000","last_fullsized":"2024-04-25T13:03:37.210084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.188491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212152+0000","last_change":"2024-04-25T13:02:44.739301+0000","last_active":"2024-04-25T13:03:37.212152+0000","last_peered":"2024-04-25T13:03:37.212152+0000","last_clean":"2024-04-25T13:03:37.212152+0000","last_became_active":"2024-04-25T13:02:44.738704+0000","last_became_peered":"2024-04-25T13:02:44.738704+0000","last_unstale":"2024-04-25T13:03:37.212152+0000","last_undegraded":"2024-04-25T13:03:37.212152+0000","last_fullsized":"2024-04-25T13:03:37.212152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:44.373522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123989+0000","last_change":"2024-04-25T13:02:44.738703+0000","last_active":"2024-04-25T13:03:37.123989+0000","last_peered":"2024-04-25T13:03:37.123989+0000","last_clean":"2024-04-25T13:03:37.123989+0000","last_became_active":"2024-04-25T13:02:44.738373+0000","last_became_peered":"2024-04-25T13:02:44.738373+0000","last_unstale":"2024-04-25T13:03:37.123989+0000","last_undegraded":"2024-04-25T13:03:37.123989+0000","last_fullsized":"2024-04-25T13:03:37.123989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:29.106615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124456+0000","last_change":"2024-04-25T13:02:44.743336+0000","last_active":"2024-04-25T13:03:37.124456+0000","last_peered":"2024-04-25T13:03:37.124456+0000","last_clean":"2024-04-25T13:03:37.124456+0000","last_became_active":"2024-04-25T13:02:44.743032+0000","last_became_peered":"2024-04-25T13:02:44.743032+0000","last_unstale":"2024-04-25T13:03:37.124456+0000","last_undegraded":"2024-04-25T13:03:37.124456+0000","last_fullsized":"2024-04-25T13:03:37.124456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:18.403138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211551+0000","last_change":"2024-04-25T13:02:44.739403+0000","last_active":"2024-04-25T13:03:37.211551+0000","last_peered":"2024-04-25T13:03:37.211551+0000","last_clean":"2024-04-25T13:03:37.211551+0000","last_became_active":"2024-04-25T13:02:44.739097+0000","last_became_peered":"2024-04-25T13:02:44.739097+0000","last_unstale":"2024-04-25T13:03:37.211551+0000","last_undegraded":"2024-04-25T13:03:37.211551+0000","last_fullsized":"2024-04-25T13:03:37.211551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:30.519343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120752+0000","last_change":"2024-04-25T13:02:44.736821+0000","last_active":"2024-04-25T13:03:37.120752+0000","last_peered":"2024-04-25T13:03:37.120752+0000","last_clean":"2024-04-25T13:03:37.120752+0000","last_became_active":"2024-04-25T13:02:44.736099+0000","last_became_peered":"2024-04-25T13:02:44.736099+0000","last_unstale":"2024-04-25T13:03:37.120752+0000","last_undegraded":"2024-04-25T13:03:37.120752+0000","last_fullsized":"2024-04-25T13:03:37.120752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:54.858845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126974+0000","last_change":"2024-04-25T13:02:44.731035+0000","last_active":"2024-04-25T13:03:37.126974+0000","last_peered":"2024-04-25T13:03:37.126974+0000","last_clean":"2024-04-25T13:03:37.126974+0000","last_became_active":"2024-04-25T13:02:44.730717+0000","last_became_peered":"2024-04-25T13:02:44.730717+0000","last_unstale":"2024-04-25T13:03:37.126974+0000","last_undegraded":"2024-04-25T13:03:37.126974+0000","last_fullsized":"2024-04-25T13:03:37.126974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:09.061183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123349+0000","last_change":"2024-04-25T13:02:44.735851+0000","last_active":"2024-04-25T13:03:37.123349+0000","last_peered":"2024-04-25T13:03:37.123349+0000","last_clean":"2024-04-25T13:03:37.123349+0000","last_became_active":"2024-04-25T13:02:44.735559+0000","last_became_peered":"2024-04-25T13:02:44.735559+0000","last_unstale":"2024-04-25T13:03:37.123349+0000","last_undegraded":"2024-04-25T13:03:37.123349+0000","last_fullsized":"2024-04-25T13:03:37.123349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.952094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:57.804844+0000","last_change":"2024-04-25T13:02:43.909800+0000","last_active":"2024-04-25T13:03:57.804844+0000","last_peered":"2024-04-25T13:03:57.804844+0000","last_clean":"2024-04-25T13:03:57.804844+0000","last_became_active":"2024-04-25T13:02:43.909463+0000","last_became_peered":"2024-04-25T13:02:43.909463+0000","last_unstale":"2024-04-25T13:03:57.804844+0000","last_undegraded":"2024-04-25T13:03:57.804844+0000","last_fullsized":"2024-04-25T13:03:57.804844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:31.495586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123734+0000","last_change":"2024-04-25T13:02:44.736710+0000","last_active":"2024-04-25T13:03:37.123734+0000","last_peered":"2024-04-25T13:03:37.123734+0000","last_clean":"2024-04-25T13:03:37.123734+0000","last_became_active":"2024-04-25T13:02:44.736385+0000","last_became_peered":"2024-04-25T13:02:44.736385+0000","last_unstale":"2024-04-25T13:03:37.123734+0000","last_undegraded":"2024-04-25T13:03:37.123734+0000","last_fullsized":"2024-04-25T13:03:37.123734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.216946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123889+0000","last_change":"2024-04-25T13:02:43.918766+0000","last_active":"2024-04-25T13:03:37.123889+0000","last_peered":"2024-04-25T13:03:37.123889+0000","last_clean":"2024-04-25T13:03:37.123889+0000","last_became_active":"2024-04-25T13:02:43.918283+0000","last_became_peered":"2024-04-25T13:02:43.918283+0000","last_unstale":"2024-04-25T13:03:37.123889+0000","last_undegraded":"2024-04-25T13:03:37.123889+0000","last_fullsized":"2024-04-25T13:03:37.123889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:17.986777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124830+0000","last_change":"2024-04-25T13:02:44.735806+0000","last_active":"2024-04-25T13:03:37.124830+0000","last_peered":"2024-04-25T13:03:37.124830+0000","last_clean":"2024-04-25T13:03:37.124830+0000","last_became_active":"2024-04-25T13:02:44.735507+0000","last_became_peered":"2024-04-25T13:02:44.735507+0000","last_unstale":"2024-04-25T13:03:37.124830+0000","last_undegraded":"2024-04-25T13:03:37.124830+0000","last_fullsized":"2024-04-25T13:03:37.124830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:22.230015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124588+0000","last_change":"2024-04-25T13:02:43.910055+0000","last_active":"2024-04-25T13:03:37.124588+0000","last_peered":"2024-04-25T13:03:37.124588+0000","last_clean":"2024-04-25T13:03:37.124588+0000","last_became_active":"2024-04-25T13:02:43.909762+0000","last_became_peered":"2024-04-25T13:02:43.909762+0000","last_unstale":"2024-04-25T13:03:37.124588+0000","last_undegraded":"2024-04-25T13:03:37.124588+0000","last_fullsized":"2024-04-25T13:03:37.124588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:47.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212007+0000","last_change":"2024-04-25T13:02:44.733253+0000","last_active":"2024-04-25T13:03:37.212007+0000","last_peered":"2024-04-25T13:03:37.212007+0000","last_clean":"2024-04-25T13:03:37.212007+0000","last_became_active":"2024-04-25T13:02:44.732917+0000","last_became_peered":"2024-04-25T13:02:44.732917+0000","last_unstale":"2024-04-25T13:03:37.212007+0000","last_undegraded":"2024-04-25T13:03:37.212007+0000","last_fullsized":"2024-04-25T13:03:37.212007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:10.380810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124091+0000","last_change":"2024-04-25T13:02:43.901490+0000","last_active":"2024-04-25T13:03:37.124091+0000","last_peered":"2024-04-25T13:03:37.124091+0000","last_clean":"2024-04-25T13:03:37.124091+0000","last_became_active":"2024-04-25T13:02:43.900807+0000","last_became_peered":"2024-04-25T13:02:43.900807+0000","last_unstale":"2024-04-25T13:03:37.124091+0000","last_undegraded":"2024-04-25T13:03:37.124091+0000","last_fullsized":"2024-04-25T13:03:37.124091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.101575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126748+0000","last_change":"2024-04-25T13:02:44.744380+0000","last_active":"2024-04-25T13:03:37.126748+0000","last_peered":"2024-04-25T13:03:37.126748+0000","last_clean":"2024-04-25T13:03:37.126748+0000","last_became_active":"2024-04-25T13:02:44.744098+0000","last_became_peered":"2024-04-25T13:02:44.744098+0000","last_unstale":"2024-04-25T13:03:37.126748+0000","last_undegraded":"2024-04-25T13:03:37.126748+0000","last_fullsized":"2024-04-25T13:03:37.126748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:53.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:55.795044+0000","last_change":"2024-04-25T13:02:43.908588+0000","last_active":"2024-04-25T13:03:55.795044+0000","last_peered":"2024-04-25T13:03:55.795044+0000","last_clean":"2024-04-25T13:03:55.795044+0000","last_became_active":"2024-04-25T13:02:43.908219+0000","last_became_peered":"2024-04-25T13:02:43.908219+0000","last_unstale":"2024-04-25T13:03:55.795044+0000","last_undegraded":"2024-04-25T13:03:55.795044+0000","last_fullsized":"2024-04-25T13:03:55.795044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:44.126509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123899+0000","last_change":"2024-04-25T13:02:44.737072+0000","last_active":"2024-04-25T13:03:37.123899+0000","last_peered":"2024-04-25T13:03:37.123899+0000","last_clean":"2024-04-25T13:03:37.123899+0000","last_became_active":"2024-04-25T13:02:44.736750+0000","last_became_peered":"2024-04-25T13:02:44.736750+0000","last_unstale":"2024-04-25T13:03:37.123899+0000","last_undegraded":"2024-04-25T13:03:37.123899+0000","last_fullsized":"2024-04-25T13:03:37.123899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:42.933740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123907+0000","last_change":"2024-04-25T13:02:43.913516+0000","last_active":"2024-04-25T13:03:37.123907+0000","last_peered":"2024-04-25T13:03:37.123907+0000","last_clean":"2024-04-25T13:03:37.123907+0000","last_became_active":"2024-04-25T13:02:43.913067+0000","last_became_peered":"2024-04-25T13:02:43.913067+0000","last_unstale":"2024-04-25T13:03:37.123907+0000","last_undegraded":"2024-04-25T13:03:37.123907+0000","last_fullsized":"2024-04-25T13:03:37.123907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:55.004598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126293+0000","last_change":"2024-04-25T13:02:44.741433+0000","last_active":"2024-04-25T13:03:37.126293+0000","last_peered":"2024-04-25T13:03:37.126293+0000","last_clean":"2024-04-25T13:03:37.126293+0000","last_became_active":"2024-04-25T13:02:44.741114+0000","last_became_peered":"2024-04-25T13:02:44.741114+0000","last_unstale":"2024-04-25T13:03:37.126293+0000","last_undegraded":"2024-04-25T13:03:37.126293+0000","last_fullsized":"2024-04-25T13:03:37.126293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:02.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211839+0000","last_change":"2024-04-25T13:02:43.900699+0000","last_active":"2024-04-25T13:03:37.211839+0000","last_peered":"2024-04-25T13:03:37.211839+0000","last_clean":"2024-04-25T13:03:37.211839+0000","last_became_active":"2024-04-25T13:02:43.900306+0000","last_became_peered":"2024-04-25T13:02:43.900306+0000","last_unstale":"2024-04-25T13:03:37.211839+0000","last_undegraded":"2024-04-25T13:03:37.211839+0000","last_fullsized":"2024-04-25T13:03:37.211839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.720365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123286+0000","last_change":"2024-04-25T13:02:44.733157+0000","last_active":"2024-04-25T13:03:37.123286+0000","last_peered":"2024-04-25T13:03:37.123286+0000","last_clean":"2024-04-25T13:03:37.123286+0000","last_became_active":"2024-04-25T13:02:44.732836+0000","last_became_peered":"2024-04-25T13:02:44.732836+0000","last_unstale":"2024-04-25T13:03:37.123286+0000","last_undegraded":"2024-04-25T13:03:37.123286+0000","last_fullsized":"2024-04-25T13:03:37.123286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:30.189708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126130+0000","last_change":"2024-04-25T13:02:43.913362+0000","last_active":"2024-04-25T13:03:37.126130+0000","last_peered":"2024-04-25T13:03:37.126130+0000","last_clean":"2024-04-25T13:03:37.126130+0000","last_became_active":"2024-04-25T13:02:43.912763+0000","last_became_peered":"2024-04-25T13:02:43.912763+0000","last_unstale":"2024-04-25T13:03:37.126130+0000","last_undegraded":"2024-04-25T13:03:37.126130+0000","last_fullsized":"2024-04-25T13:03:37.126130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:12.216165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126796+0000","last_change":"2024-04-25T13:02:44.732662+0000","last_active":"2024-04-25T13:03:37.126796+0000","last_peered":"2024-04-25T13:03:37.126796+0000","last_clean":"2024-04-25T13:03:37.126796+0000","last_became_active":"2024-04-25T13:02:44.732319+0000","last_became_peered":"2024-04-25T13:02:44.732319+0000","last_unstale":"2024-04-25T13:03:37.126796+0000","last_undegraded":"2024-04-25T13:03:37.126796+0000","last_fullsized":"2024-04-25T13:03:37.126796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:51.683916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":161,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:58.799023+0000","last_change":"2024-04-25T13:02:43.911019+0000","last_active":"2024-04-25T13:03:58.799023+0000","last_peered":"2024-04-25T13:03:58.799023+0000","last_clean":"2024-04-25T13:03:58.799023+0000","last_became_active":"2024-04-25T13:02:43.910707+0000","last_became_peered":"2024-04-25T13:02:43.910707+0000","last_unstale":"2024-04-25T13:03:58.799023+0000","last_undegraded":"2024-04-25T13:03:58.799023+0000","last_fullsized":"2024-04-25T13:03:58.799023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:07.042580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212101+0000","last_change":"2024-04-25T13:02:44.732791+0000","last_active":"2024-04-25T13:03:37.212101+0000","last_peered":"2024-04-25T13:03:37.212101+0000","last_clean":"2024-04-25T13:03:37.212101+0000","last_became_active":"2024-04-25T13:02:44.732484+0000","last_became_peered":"2024-04-25T13:02:44.732484+0000","last_unstale":"2024-04-25T13:03:37.212101+0000","last_undegraded":"2024-04-25T13:03:37.212101+0000","last_fullsized":"2024-04-25T13:03:37.212101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:36.164774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123437+0000","last_change":"2024-04-25T13:02:43.912447+0000","last_active":"2024-04-25T13:03:37.123437+0000","last_peered":"2024-04-25T13:03:37.123437+0000","last_clean":"2024-04-25T13:03:37.123437+0000","last_became_active":"2024-04-25T13:02:43.912091+0000","last_became_peered":"2024-04-25T13:02:43.912091+0000","last_unstale":"2024-04-25T13:03:37.123437+0000","last_undegraded":"2024-04-25T13:03:37.123437+0000","last_fullsized":"2024-04-25T13:03:37.123437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:41.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.125144+0000","last_change":"2024-04-25T13:02:44.739021+0000","last_active":"2024-04-25T13:03:37.125144+0000","last_peered":"2024-04-25T13:03:37.125144+0000","last_clean":"2024-04-25T13:03:37.125144+0000","last_became_active":"2024-04-25T13:02:44.737945+0000","last_became_peered":"2024-04-25T13:02:44.737945+0000","last_unstale":"2024-04-25T13:03:37.125144+0000","last_undegraded":"2024-04-25T13:03:37.125144+0000","last_fullsized":"2024-04-25T13:03:37.125144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:09.405021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126867+0000","last_change":"2024-04-25T13:02:43.909061+0000","last_active":"2024-04-25T13:03:37.126867+0000","last_peered":"2024-04-25T13:03:37.126867+0000","last_clean":"2024-04-25T13:03:37.126867+0000","last_became_active":"2024-04-25T13:02:43.908739+0000","last_became_peered":"2024-04-25T13:02:43.908739+0000","last_unstale":"2024-04-25T13:03:37.126867+0000","last_undegraded":"2024-04-25T13:03:37.126867+0000","last_fullsized":"2024-04-25T13:03:37.126867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:25.720808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212132+0000","last_change":"2024-04-25T13:02:44.734346+0000","last_active":"2024-04-25T13:03:37.212132+0000","last_peered":"2024-04-25T13:03:37.212132+0000","last_clean":"2024-04-25T13:03:37.212132+0000","last_became_active":"2024-04-25T13:02:44.734045+0000","last_became_peered":"2024-04-25T13:02:44.734045+0000","last_unstale":"2024-04-25T13:03:37.212132+0000","last_undegraded":"2024-04-25T13:03:37.212132+0000","last_fullsized":"2024-04-25T13:03:37.212132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:42.137455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":191,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:58.801664+0000","last_change":"2024-04-25T13:02:43.910212+0000","last_active":"2024-04-25T13:03:58.801664+0000","last_peered":"2024-04-25T13:03:58.801664+0000","last_clean":"2024-04-25T13:03:58.801664+0000","last_became_active":"2024-04-25T13:02:43.909873+0000","last_became_peered":"2024-04-25T13:02:43.909873+0000","last_unstale":"2024-04-25T13:03:58.801664+0000","last_undegraded":"2024-04-25T13:03:58.801664+0000","last_fullsized":"2024-04-25T13:03:58.801664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.716434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":144,"num_read_kb":129,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124406+0000","last_change":"2024-04-25T13:02:44.731633+0000","last_active":"2024-04-25T13:03:37.124406+0000","last_peered":"2024-04-25T13:03:37.124406+0000","last_clean":"2024-04-25T13:03:37.124406+0000","last_became_active":"2024-04-25T13:02:44.731360+0000","last_became_peered":"2024-04-25T13:02:44.731360+0000","last_unstale":"2024-04-25T13:03:37.124406+0000","last_undegraded":"2024-04-25T13:03:37.124406+0000","last_fullsized":"2024-04-25T13:03:37.124406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:12.960787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.137667+0000","last_change":"2024-04-25T13:03:37.137667+0000","last_active":"2024-04-25T13:03:37.137667+0000","last_peered":"2024-04-25T13:03:37.137667+0000","last_clean":"2024-04-25T13:03:37.137667+0000","last_became_active":"2024-04-25T13:03:37.125779+0000","last_became_peered":"2024-04-25T13:03:37.125779+0000","last_unstale":"2024-04-25T13:03:37.137667+0000","last_undegraded":"2024-04-25T13:03:37.137667+0000","last_fullsized":"2024-04-25T13:03:37.137667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:26.963757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212330+0000","last_change":"2024-04-25T13:02:44.742431+0000","last_active":"2024-04-25T13:03:37.212330+0000","last_peered":"2024-04-25T13:03:37.212330+0000","last_clean":"2024-04-25T13:03:37.212330+0000","last_became_active":"2024-04-25T13:02:44.742120+0000","last_became_peered":"2024-04-25T13:02:44.742120+0000","last_unstale":"2024-04-25T13:03:37.212330+0000","last_undegraded":"2024-04-25T13:03:37.212330+0000","last_fullsized":"2024-04-25T13:03:37.212330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:08.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126257+0000","last_change":"2024-04-25T13:02:43.911342+0000","last_active":"2024-04-25T13:03:37.126257+0000","last_peered":"2024-04-25T13:03:37.126257+0000","last_clean":"2024-04-25T13:03:37.126257+0000","last_became_active":"2024-04-25T13:02:43.910741+0000","last_became_peered":"2024-04-25T13:02:43.910741+0000","last_unstale":"2024-04-25T13:03:37.126257+0000","last_undegraded":"2024-04-25T13:03:37.126257+0000","last_fullsized":"2024-04-25T13:03:37.126257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:28.464712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123637+0000","last_change":"2024-04-25T13:02:44.736639+0000","last_active":"2024-04-25T13:03:37.123637+0000","last_peered":"2024-04-25T13:03:37.123637+0000","last_clean":"2024-04-25T13:03:37.123637+0000","last_became_active":"2024-04-25T13:02:44.736285+0000","last_became_peered":"2024-04-25T13:02:44.736285+0000","last_unstale":"2024-04-25T13:03:37.123637+0000","last_undegraded":"2024-04-25T13:03:37.123637+0000","last_fullsized":"2024-04-25T13:03:37.123637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.840298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:55.817876+0000","last_change":"2024-04-25T13:02:43.906893+0000","last_active":"2024-04-25T13:03:55.817876+0000","last_peered":"2024-04-25T13:03:55.817876+0000","last_clean":"2024-04-25T13:03:55.817876+0000","last_became_active":"2024-04-25T13:02:43.906538+0000","last_became_peered":"2024-04-25T13:02:43.906538+0000","last_unstale":"2024-04-25T13:03:55.817876+0000","last_undegraded":"2024-04-25T13:03:55.817876+0000","last_fullsized":"2024-04-25T13:03:55.817876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:10.601792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.122894+0000","last_change":"2024-04-25T13:02:44.741092+0000","last_active":"2024-04-25T13:03:37.122894+0000","last_peered":"2024-04-25T13:03:37.122894+0000","last_clean":"2024-04-25T13:03:37.122894+0000","last_became_active":"2024-04-25T13:02:44.740755+0000","last_became_peered":"2024-04-25T13:02:44.740755+0000","last_unstale":"2024-04-25T13:03:37.122894+0000","last_undegraded":"2024-04-25T13:03:37.122894+0000","last_fullsized":"2024-04-25T13:03:37.122894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:49.124211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123602+0000","last_change":"2024-04-25T13:02:43.909572+0000","last_active":"2024-04-25T13:03:37.123602+0000","last_peered":"2024-04-25T13:03:37.123602+0000","last_clean":"2024-04-25T13:03:37.123602+0000","last_became_active":"2024-04-25T13:02:43.909109+0000","last_became_peered":"2024-04-25T13:02:43.909109+0000","last_unstale":"2024-04-25T13:03:37.123602+0000","last_undegraded":"2024-04-25T13:03:37.123602+0000","last_fullsized":"2024-04-25T13:03:37.123602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.505599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126256+0000","last_change":"2024-04-25T13:02:44.740972+0000","last_active":"2024-04-25T13:03:37.126256+0000","last_peered":"2024-04-25T13:03:37.126256+0000","last_clean":"2024-04-25T13:03:37.126256+0000","last_became_active":"2024-04-25T13:02:44.740620+0000","last_became_peered":"2024-04-25T13:02:44.740620+0000","last_unstale":"2024-04-25T13:03:37.126256+0000","last_undegraded":"2024-04-25T13:03:37.126256+0000","last_fullsized":"2024-04-25T13:03:37.126256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:45.787763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124498+0000","last_change":"2024-04-25T13:02:43.901205+0000","last_active":"2024-04-25T13:03:37.124498+0000","last_peered":"2024-04-25T13:03:37.124498+0000","last_clean":"2024-04-25T13:03:37.124498+0000","last_became_active":"2024-04-25T13:02:43.900796+0000","last_became_peered":"2024-04-25T13:02:43.900796+0000","last_unstale":"2024-04-25T13:03:37.124498+0000","last_undegraded":"2024-04-25T13:03:37.124498+0000","last_fullsized":"2024-04-25T13:03:37.124498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:14.020387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211277+0000","last_change":"2024-04-25T13:03:37.211277+0000","last_active":"2024-04-25T13:03:37.211277+0000","last_peered":"2024-04-25T13:03:37.211277+0000","last_clean":"2024-04-25T13:03:37.211277+0000","last_became_active":"2024-04-25T13:03:37.210854+0000","last_became_peered":"2024-04-25T13:03:37.210854+0000","last_unstale":"2024-04-25T13:03:37.211277+0000","last_undegraded":"2024-04-25T13:03:37.211277+0000","last_fullsized":"2024-04-25T13:03:37.211277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.569591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126938+0000","last_change":"2024-04-25T13:02:43.911812+0000","last_active":"2024-04-25T13:03:37.126938+0000","last_peered":"2024-04-25T13:03:37.126938+0000","last_clean":"2024-04-25T13:03:37.126938+0000","last_became_active":"2024-04-25T13:02:43.911404+0000","last_became_peered":"2024-04-25T13:02:43.911404+0000","last_unstale":"2024-04-25T13:03:37.126938+0000","last_undegraded":"2024-04-25T13:03:37.126938+0000","last_fullsized":"2024-04-25T13:03:37.126938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:17.210980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123056+0000","last_change":"2024-04-25T13:02:44.732546+0000","last_active":"2024-04-25T13:03:37.123056+0000","last_peered":"2024-04-25T13:03:37.123056+0000","last_clean":"2024-04-25T13:03:37.123056+0000","last_became_active":"2024-04-25T13:02:44.732238+0000","last_became_peered":"2024-04-25T13:02:44.732238+0000","last_unstale":"2024-04-25T13:03:37.123056+0000","last_undegraded":"2024-04-25T13:03:37.123056+0000","last_fullsized":"2024-04-25T13:03:37.123056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.293939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:58.806376+0000","last_change":"2024-04-25T13:02:43.910620+0000","last_active":"2024-04-25T13:03:58.806376+0000","last_peered":"2024-04-25T13:03:58.806376+0000","last_clean":"2024-04-25T13:03:58.806376+0000","last_became_active":"2024-04-25T13:02:43.910325+0000","last_became_peered":"2024-04-25T13:02:43.910325+0000","last_unstale":"2024-04-25T13:03:58.806376+0000","last_undegraded":"2024-04-25T13:03:58.806376+0000","last_fullsized":"2024-04-25T13:03:58.806376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.185209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123028+0000","last_change":"2024-04-25T13:02:44.737134+0000","last_active":"2024-04-25T13:03:37.123028+0000","last_peered":"2024-04-25T13:03:37.123028+0000","last_clean":"2024-04-25T13:03:37.123028+0000","last_became_active":"2024-04-25T13:02:44.736790+0000","last_became_peered":"2024-04-25T13:02:44.736790+0000","last_unstale":"2024-04-25T13:03:37.123028+0000","last_undegraded":"2024-04-25T13:03:37.123028+0000","last_fullsized":"2024-04-25T13:03:37.123028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:02.680752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211874+0000","last_change":"2024-04-25T13:03:37.211039+0000","last_active":"2024-04-25T13:03:37.211874+0000","last_peered":"2024-04-25T13:03:37.211874+0000","last_clean":"2024-04-25T13:03:37.211874+0000","last_became_active":"2024-04-25T13:03:37.210634+0000","last_became_peered":"2024-04-25T13:03:37.210634+0000","last_unstale":"2024-04-25T13:03:37.211874+0000","last_undegraded":"2024-04-25T13:03:37.211874+0000","last_fullsized":"2024-04-25T13:03:37.211874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:43.772416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:57.820605+0000","last_change":"2024-04-25T13:02:43.920678+0000","last_active":"2024-04-25T13:03:57.820605+0000","last_peered":"2024-04-25T13:03:57.820605+0000","last_clean":"2024-04-25T13:03:57.820605+0000","last_became_active":"2024-04-25T13:02:43.920338+0000","last_became_peered":"2024-04-25T13:02:43.920338+0000","last_unstale":"2024-04-25T13:03:57.820605+0000","last_undegraded":"2024-04-25T13:03:57.820605+0000","last_fullsized":"2024-04-25T13:03:57.820605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123232+0000","last_change":"2024-04-25T13:02:44.736655+0000","last_active":"2024-04-25T13:03:37.123232+0000","last_peered":"2024-04-25T13:03:37.123232+0000","last_clean":"2024-04-25T13:03:37.123232+0000","last_became_active":"2024-04-25T13:02:44.736340+0000","last_became_peered":"2024-04-25T13:02:44.736340+0000","last_unstale":"2024-04-25T13:03:37.123232+0000","last_undegraded":"2024-04-25T13:03:37.123232+0000","last_fullsized":"2024-04-25T13:03:37.123232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:35.129606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.210381+0000","last_change":"2024-04-25T13:02:43.903179+0000","last_active":"2024-04-25T13:03:37.210381+0000","last_peered":"2024-04-25T13:03:37.210381+0000","last_clean":"2024-04-25T13:03:37.210381+0000","last_became_active":"2024-04-25T13:02:43.902799+0000","last_became_peered":"2024-04-25T13:02:43.902799+0000","last_unstale":"2024-04-25T13:03:37.210381+0000","last_undegraded":"2024-04-25T13:03:37.210381+0000","last_fullsized":"2024-04-25T13:03:37.210381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.472772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123636+0000","last_change":"2024-04-25T13:01:33.625244+0000","last_active":"2024-04-25T13:03:37.123636+0000","last_peered":"2024-04-25T13:03:37.123636+0000","last_clean":"2024-04-25T13:03:37.123636+0000","last_became_active":"2024-04-25T13:01:33.624760+0000","last_became_peered":"2024-04-25T13:01:33.624760+0000","last_unstale":"2024-04-25T13:03:37.123636+0000","last_undegraded":"2024-04-25T13:03:37.123636+0000","last_fullsized":"2024-04-25T13:03:37.123636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_clean_scrub_stamp":"2024-04-25T12:58:01.939677+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:21.636432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211698+0000","last_change":"2024-04-25T13:02:44.732711+0000","last_active":"2024-04-25T13:03:37.211698+0000","last_peered":"2024-04-25T13:03:37.211698+0000","last_clean":"2024-04-25T13:03:37.211698+0000","last_became_active":"2024-04-25T13:02:44.732394+0000","last_became_peered":"2024-04-25T13:02:44.732394+0000","last_unstale":"2024-04-25T13:03:37.211698+0000","last_undegraded":"2024-04-25T13:03:37.211698+0000","last_fullsized":"2024-04-25T13:03:37.211698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:02.601328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124195+0000","last_change":"2024-04-25T13:02:43.908773+0000","last_active":"2024-04-25T13:03:37.124195+0000","last_peered":"2024-04-25T13:03:37.124195+0000","last_clean":"2024-04-25T13:03:37.124195+0000","last_became_active":"2024-04-25T13:02:43.908440+0000","last_became_peered":"2024-04-25T13:02:43.908440+0000","last_unstale":"2024-04-25T13:03:37.124195+0000","last_undegraded":"2024-04-25T13:03:37.124195+0000","last_fullsized":"2024-04-25T13:03:37.124195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:49.565391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.143437+0000","last_change":"2024-04-25T13:03:37.143437+0000","last_active":"2024-04-25T13:03:37.143437+0000","last_peered":"2024-04-25T13:03:37.143437+0000","last_clean":"2024-04-25T13:03:37.143437+0000","last_became_active":"2024-04-25T13:03:37.127801+0000","last_became_peered":"2024-04-25T13:03:37.127801+0000","last_unstale":"2024-04-25T13:03:37.143437+0000","last_undegraded":"2024-04-25T13:03:37.143437+0000","last_fullsized":"2024-04-25T13:03:37.143437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:47.756708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212152+0000","last_change":"2024-04-25T13:02:44.730629+0000","last_active":"2024-04-25T13:03:37.212152+0000","last_peered":"2024-04-25T13:03:37.212152+0000","last_clean":"2024-04-25T13:03:37.212152+0000","last_became_active":"2024-04-25T13:02:44.730288+0000","last_became_peered":"2024-04-25T13:02:44.730288+0000","last_unstale":"2024-04-25T13:03:37.212152+0000","last_undegraded":"2024-04-25T13:03:37.212152+0000","last_fullsized":"2024-04-25T13:03:37.212152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:26.255183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.127606+0000","last_change":"2024-04-25T13:02:43.918791+0000","last_active":"2024-04-25T13:03:37.127606+0000","last_peered":"2024-04-25T13:03:37.127606+0000","last_clean":"2024-04-25T13:03:37.127606+0000","last_became_active":"2024-04-25T13:02:43.918454+0000","last_became_peered":"2024-04-25T13:02:43.918454+0000","last_unstale":"2024-04-25T13:03:37.127606+0000","last_undegraded":"2024-04-25T13:03:37.127606+0000","last_fullsized":"2024-04-25T13:03:37.127606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.137235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123965+0000","last_change":"2024-04-25T13:02:44.740324+0000","last_active":"2024-04-25T13:03:37.123965+0000","last_peered":"2024-04-25T13:03:37.123965+0000","last_clean":"2024-04-25T13:03:37.123965+0000","last_became_active":"2024-04-25T13:02:44.739993+0000","last_became_peered":"2024-04-25T13:02:44.739993+0000","last_unstale":"2024-04-25T13:03:37.123965+0000","last_undegraded":"2024-04-25T13:03:37.123965+0000","last_fullsized":"2024-04-25T13:03:37.123965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.207392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.209860+0000","last_change":"2024-04-25T13:02:43.903356+0000","last_active":"2024-04-25T13:03:37.209860+0000","last_peered":"2024-04-25T13:03:37.209860+0000","last_clean":"2024-04-25T13:03:37.209860+0000","last_became_active":"2024-04-25T13:02:43.903046+0000","last_became_peered":"2024-04-25T13:02:43.903046+0000","last_unstale":"2024-04-25T13:03:37.209860+0000","last_undegraded":"2024-04-25T13:03:37.209860+0000","last_fullsized":"2024-04-25T13:03:37.209860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.514373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.125168+0000","last_change":"2024-04-25T13:02:44.745033+0000","last_active":"2024-04-25T13:03:37.125168+0000","last_peered":"2024-04-25T13:03:37.125168+0000","last_clean":"2024-04-25T13:03:37.125168+0000","last_became_active":"2024-04-25T13:02:44.744718+0000","last_became_peered":"2024-04-25T13:02:44.744718+0000","last_unstale":"2024-04-25T13:03:37.125168+0000","last_undegraded":"2024-04-25T13:03:37.125168+0000","last_fullsized":"2024-04-25T13:03:37.125168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:26.736807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211810+0000","last_change":"2024-04-25T13:02:43.909366+0000","last_active":"2024-04-25T13:03:37.211810+0000","last_peered":"2024-04-25T13:03:37.211810+0000","last_clean":"2024-04-25T13:03:37.211810+0000","last_became_active":"2024-04-25T13:02:43.908968+0000","last_became_peered":"2024-04-25T13:02:43.908968+0000","last_unstale":"2024-04-25T13:03:37.211810+0000","last_undegraded":"2024-04-25T13:03:37.211810+0000","last_fullsized":"2024-04-25T13:03:37.211810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:36.915466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123714+0000","last_change":"2024-04-25T13:02:44.730481+0000","last_active":"2024-04-25T13:03:37.123714+0000","last_peered":"2024-04-25T13:03:37.123714+0000","last_clean":"2024-04-25T13:03:37.123714+0000","last_became_active":"2024-04-25T13:02:44.730163+0000","last_became_peered":"2024-04-25T13:02:44.730163+0000","last_unstale":"2024-04-25T13:03:37.123714+0000","last_undegraded":"2024-04-25T13:03:37.123714+0000","last_fullsized":"2024-04-25T13:03:37.123714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:12.207840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123352+0000","last_change":"2024-04-25T13:02:43.922262+0000","last_active":"2024-04-25T13:03:37.123352+0000","last_peered":"2024-04-25T13:03:37.123352+0000","last_clean":"2024-04-25T13:03:37.123352+0000","last_became_active":"2024-04-25T13:02:43.921356+0000","last_became_peered":"2024-04-25T13:02:43.921356+0000","last_unstale":"2024-04-25T13:03:37.123352+0000","last_undegraded":"2024-04-25T13:03:37.123352+0000","last_fullsized":"2024-04-25T13:03:37.123352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:27.446112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120335+0000","last_change":"2024-04-25T13:02:44.735879+0000","last_active":"2024-04-25T13:03:37.120335+0000","last_peered":"2024-04-25T13:03:37.120335+0000","last_clean":"2024-04-25T13:03:37.120335+0000","last_became_active":"2024-04-25T13:02:44.735574+0000","last_became_peered":"2024-04-25T13:02:44.735574+0000","last_unstale":"2024-04-25T13:03:37.120335+0000","last_undegraded":"2024-04-25T13:03:37.120335+0000","last_fullsized":"2024-04-25T13:03:37.120335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:14.094813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123231+0000","last_change":"2024-04-25T13:02:43.912250+0000","last_active":"2024-04-25T13:03:37.123231+0000","last_peered":"2024-04-25T13:03:37.123231+0000","last_clean":"2024-04-25T13:03:37.123231+0000","last_became_active":"2024-04-25T13:02:43.911945+0000","last_became_peered":"2024-04-25T13:02:43.911945+0000","last_unstale":"2024-04-25T13:03:37.123231+0000","last_undegraded":"2024-04-25T13:03:37.123231+0000","last_fullsized":"2024-04-25T13:03:37.123231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:51.872195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123175+0000","last_change":"2024-04-25T13:02:44.739184+0000","last_active":"2024-04-25T13:03:37.123175+0000","last_peered":"2024-04-25T13:03:37.123175+0000","last_clean":"2024-04-25T13:03:37.123175+0000","last_became_active":"2024-04-25T13:02:44.738862+0000","last_became_peered":"2024-04-25T13:02:44.738862+0000","last_unstale":"2024-04-25T13:03:37.123175+0000","last_undegraded":"2024-04-25T13:03:37.123175+0000","last_fullsized":"2024-04-25T13:03:37.123175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.605730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.215691+0000","last_change":"2024-04-25T13:03:37.215691+0000","last_active":"2024-04-25T13:03:37.215691+0000","last_peered":"2024-04-25T13:03:37.215691+0000","last_clean":"2024-04-25T13:03:37.215691+0000","last_became_active":"2024-04-25T13:03:37.211345+0000","last_became_peered":"2024-04-25T13:03:37.211345+0000","last_unstale":"2024-04-25T13:03:37.215691+0000","last_undegraded":"2024-04-25T13:03:37.215691+0000","last_fullsized":"2024-04-25T13:03:37.215691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:15.904620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124160+0000","last_change":"2024-04-25T13:02:44.738410+0000","last_active":"2024-04-25T13:03:37.124160+0000","last_peered":"2024-04-25T13:03:37.124160+0000","last_clean":"2024-04-25T13:03:37.124160+0000","last_became_active":"2024-04-25T13:02:44.738091+0000","last_became_peered":"2024-04-25T13:02:44.738091+0000","last_unstale":"2024-04-25T13:03:37.124160+0000","last_undegraded":"2024-04-25T13:03:37.124160+0000","last_fullsized":"2024-04-25T13:03:37.124160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.737907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:58.799839+0000","last_change":"2024-04-25T13:02:43.914563+0000","last_active":"2024-04-25T13:03:58.799839+0000","last_peered":"2024-04-25T13:03:58.799839+0000","last_clean":"2024-04-25T13:03:58.799839+0000","last_became_active":"2024-04-25T13:02:43.914274+0000","last_became_peered":"2024-04-25T13:02:43.914274+0000","last_unstale":"2024-04-25T13:03:58.799839+0000","last_undegraded":"2024-04-25T13:03:58.799839+0000","last_fullsized":"2024-04-25T13:03:58.799839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124115+0000","last_change":"2024-04-25T13:02:44.730098+0000","last_active":"2024-04-25T13:03:37.124115+0000","last_peered":"2024-04-25T13:03:37.124115+0000","last_clean":"2024-04-25T13:03:37.124115+0000","last_became_active":"2024-04-25T13:02:44.729656+0000","last_became_peered":"2024-04-25T13:02:44.729656+0000","last_unstale":"2024-04-25T13:03:37.124115+0000","last_undegraded":"2024-04-25T13:03:37.124115+0000","last_fullsized":"2024-04-25T13:03:37.124115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.344967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126628+0000","last_change":"2024-04-25T13:02:43.911593+0000","last_active":"2024-04-25T13:03:37.126628+0000","last_peered":"2024-04-25T13:03:37.126628+0000","last_clean":"2024-04-25T13:03:37.126628+0000","last_became_active":"2024-04-25T13:02:43.910484+0000","last_became_peered":"2024-04-25T13:02:43.910484+0000","last_unstale":"2024-04-25T13:03:37.126628+0000","last_undegraded":"2024-04-25T13:03:37.126628+0000","last_fullsized":"2024-04-25T13:03:37.126628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:52.257269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.132602+0000","last_change":"2024-04-25T13:03:37.132602+0000","last_active":"2024-04-25T13:03:37.132602+0000","last_peered":"2024-04-25T13:03:37.132602+0000","last_clean":"2024-04-25T13:03:37.132602+0000","last_became_active":"2024-04-25T13:03:37.128042+0000","last_became_peered":"2024-04-25T13:03:37.128042+0000","last_unstale":"2024-04-25T13:03:37.132602+0000","last_undegraded":"2024-04-25T13:03:37.132602+0000","last_fullsized":"2024-04-25T13:03:37.132602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.577684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212472+0000","last_change":"2024-04-25T13:02:43.921129+0000","last_active":"2024-04-25T13:03:37.212472+0000","last_peered":"2024-04-25T13:03:37.212472+0000","last_clean":"2024-04-25T13:03:37.212472+0000","last_became_active":"2024-04-25T13:02:43.920768+0000","last_became_peered":"2024-04-25T13:02:43.920768+0000","last_unstale":"2024-04-25T13:03:37.212472+0000","last_undegraded":"2024-04-25T13:03:37.212472+0000","last_fullsized":"2024-04-25T13:03:37.212472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:49.178313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123113+0000","last_change":"2024-04-25T13:02:44.731934+0000","last_active":"2024-04-25T13:03:37.123113+0000","last_peered":"2024-04-25T13:03:37.123113+0000","last_clean":"2024-04-25T13:03:37.123113+0000","last_became_active":"2024-04-25T13:02:44.731624+0000","last_became_peered":"2024-04-25T13:02:44.731624+0000","last_unstale":"2024-04-25T13:03:37.123113+0000","last_undegraded":"2024-04-25T13:03:37.123113+0000","last_fullsized":"2024-04-25T13:03:37.123113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.809419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120309+0000","last_change":"2024-04-25T13:02:44.736982+0000","last_active":"2024-04-25T13:03:37.120309+0000","last_peered":"2024-04-25T13:03:37.120309+0000","last_clean":"2024-04-25T13:03:37.120309+0000","last_became_active":"2024-04-25T13:02:44.735127+0000","last_became_peered":"2024-04-25T13:02:44.735127+0000","last_unstale":"2024-04-25T13:03:37.120309+0000","last_undegraded":"2024-04-25T13:03:37.120309+0000","last_fullsized":"2024-04-25T13:03:37.120309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:06.827686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.213302+0000","last_change":"2024-04-25T13:03:37.213302+0000","last_active":"2024-04-25T13:03:37.213302+0000","last_peered":"2024-04-25T13:03:37.213302+0000","last_clean":"2024-04-25T13:03:37.213302+0000","last_became_active":"2024-04-25T13:03:37.212873+0000","last_became_peered":"2024-04-25T13:03:37.212873+0000","last_unstale":"2024-04-25T13:03:37.213302+0000","last_undegraded":"2024-04-25T13:03:37.213302+0000","last_fullsized":"2024-04-25T13:03:37.213302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:36.988761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123122+0000","last_change":"2024-04-25T13:02:44.739660+0000","last_active":"2024-04-25T13:03:37.123122+0000","last_peered":"2024-04-25T13:03:37.123122+0000","last_clean":"2024-04-25T13:03:37.123122+0000","last_became_active":"2024-04-25T13:02:44.738553+0000","last_became_peered":"2024-04-25T13:02:44.738553+0000","last_unstale":"2024-04-25T13:03:37.123122+0000","last_undegraded":"2024-04-25T13:03:37.123122+0000","last_fullsized":"2024-04-25T13:03:37.123122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:07.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.213046+0000","last_change":"2024-04-25T13:03:37.213046+0000","last_active":"2024-04-25T13:03:37.213046+0000","last_peered":"2024-04-25T13:03:37.213046+0000","last_clean":"2024-04-25T13:03:37.213046+0000","last_became_active":"2024-04-25T13:03:37.212649+0000","last_became_peered":"2024-04-25T13:03:37.212649+0000","last_unstale":"2024-04-25T13:03:37.213046+0000","last_undegraded":"2024-04-25T13:03:37.213046+0000","last_fullsized":"2024-04-25T13:03:37.213046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:57.193414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212110+0000","last_change":"2024-04-25T13:02:44.739324+0000","last_active":"2024-04-25T13:03:37.212110+0000","last_peered":"2024-04-25T13:03:37.212110+0000","last_clean":"2024-04-25T13:03:37.212110+0000","last_became_active":"2024-04-25T13:02:44.738994+0000","last_became_peered":"2024-04-25T13:02:44.738994+0000","last_unstale":"2024-04-25T13:03:37.212110+0000","last_undegraded":"2024-04-25T13:03:37.212110+0000","last_fullsized":"2024-04-25T13:03:37.212110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:50.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120646+0000","last_change":"2024-04-25T13:02:44.738075+0000","last_active":"2024-04-25T13:03:37.120646+0000","last_peered":"2024-04-25T13:03:37.120646+0000","last_clean":"2024-04-25T13:03:37.120646+0000","last_became_active":"2024-04-25T13:02:44.736616+0000","last_became_peered":"2024-04-25T13:02:44.736616+0000","last_unstale":"2024-04-25T13:03:37.120646+0000","last_undegraded":"2024-04-25T13:03:37.120646+0000","last_fullsized":"2024-04-25T13:03:37.120646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:14.875245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123380+0000","last_change":"2024-04-25T13:02:44.730926+0000","last_active":"2024-04-25T13:03:37.123380+0000","last_peered":"2024-04-25T13:03:37.123380+0000","last_clean":"2024-04-25T13:03:37.123380+0000","last_became_active":"2024-04-25T13:02:44.730154+0000","last_became_peered":"2024-04-25T13:02:44.730154+0000","last_unstale":"2024-04-25T13:03:37.123380+0000","last_undegraded":"2024-04-25T13:03:37.123380+0000","last_fullsized":"2024-04-25T13:03:37.123380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:07.361869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.128334+0000","last_change":"2024-04-25T13:03:37.128334+0000","last_active":"2024-04-25T13:03:37.128334+0000","last_peered":"2024-04-25T13:03:37.128334+0000","last_clean":"2024-04-25T13:03:37.128334+0000","last_became_active":"2024-04-25T13:03:37.127873+0000","last_became_peered":"2024-04-25T13:03:37.127873+0000","last_unstale":"2024-04-25T13:03:37.128334+0000","last_undegraded":"2024-04-25T13:03:37.128334+0000","last_fullsized":"2024-04-25T13:03:37.128334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.911113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126280+0000","last_change":"2024-04-25T13:02:44.735555+0000","last_active":"2024-04-25T13:03:37.126280+0000","last_peered":"2024-04-25T13:03:37.126280+0000","last_clean":"2024-04-25T13:03:37.126280+0000","last_became_active":"2024-04-25T13:02:44.735238+0000","last_became_peered":"2024-04-25T13:02:44.735238+0000","last_unstale":"2024-04-25T13:03:37.126280+0000","last_undegraded":"2024-04-25T13:03:37.126280+0000","last_fullsized":"2024-04-25T13:03:37.126280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.611943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123114+0000","last_change":"2024-04-25T13:02:44.730959+0000","last_active":"2024-04-25T13:03:37.123114+0000","last_peered":"2024-04-25T13:03:37.123114+0000","last_clean":"2024-04-25T13:03:37.123114+0000","last_became_active":"2024-04-25T13:02:44.730583+0000","last_became_peered":"2024-04-25T13:02:44.730583+0000","last_unstale":"2024-04-25T13:03:37.123114+0000","last_undegraded":"2024-04-25T13:03:37.123114+0000","last_fullsized":"2024-04-25T13:03:37.123114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.527837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211809+0000","last_change":"2024-04-25T13:02:44.734174+0000","last_active":"2024-04-25T13:03:37.211809+0000","last_peered":"2024-04-25T13:03:37.211809+0000","last_clean":"2024-04-25T13:03:37.211809+0000","last_became_active":"2024-04-25T13:02:44.733785+0000","last_became_peered":"2024-04-25T13:02:44.733785+0000","last_unstale":"2024-04-25T13:03:37.211809+0000","last_undegraded":"2024-04-25T13:03:37.211809+0000","last_fullsized":"2024-04-25T13:03:37.211809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:39.641206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212228+0000","last_change":"2024-04-25T13:02:44.740058+0000","last_active":"2024-04-25T13:03:37.212228+0000","last_peered":"2024-04-25T13:03:37.212228+0000","last_clean":"2024-04-25T13:03:37.212228+0000","last_became_active":"2024-04-25T13:02:44.739716+0000","last_became_peered":"2024-04-25T13:02:44.739716+0000","last_unstale":"2024-04-25T13:03:37.212228+0000","last_undegraded":"2024-04-25T13:03:37.212228+0000","last_fullsized":"2024-04-25T13:03:37.212228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:21.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120271+0000","last_change":"2024-04-25T13:02:44.731335+0000","last_active":"2024-04-25T13:03:37.120271+0000","last_peered":"2024-04-25T13:03:37.120271+0000","last_clean":"2024-04-25T13:03:37.120271+0000","last_became_active":"2024-04-25T13:02:44.731017+0000","last_became_peered":"2024-04-25T13:02:44.731017+0000","last_unstale":"2024-04-25T13:03:37.120271+0000","last_undegraded":"2024-04-25T13:03:37.120271+0000","last_fullsized":"2024-04-25T13:03:37.120271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:29.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":288217,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65334,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":721,"num_read_kb":433,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":196002,"data_compressed":5601,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":460141,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1109902,"data_compressed":44333,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":460008,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":460157,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1052396,"data_compressed":42466,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":459608,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":519433,"data_compressed":7692,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":460947,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":519585,"data_compressed":7692,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":459316,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459469,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":1052074,"data_compressed":42466,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60664,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60816,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60648,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:04:03.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:04:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:03 smithi104 ceph-mon[32726]: from='mgr.14152 172.21.15.38:0/1178744329' entity='mgr.x' 2024-04-25T13:04:03.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:03 smithi104 ceph-mon[32726]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:04.032 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:05.198 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:05 smithi038 ceph-mon[29400]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:05.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:05 smithi104 ceph-mon[32726]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:05.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:05 smithi178 ceph-mon[33233]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 -- 172.21.15.38:0/387690683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418108a50 msgr2=0x7f5418108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 --2- 172.21.15.38:0/387690683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418108a50 0x7f5418108e30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5408009990 tx=0x7f5408009ca0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 -- 172.21.15.38:0/387690683 shutdown_connections 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 --2- 172.21.15.38:0/387690683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 0x7f541810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 --2- 172.21.15.38:0/387690683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418108a50 0x7f5418108e30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 --2- 172.21.15.38:0/387690683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418102a90 0x7f5418102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 -- 172.21.15.38:0/387690683 >> 172.21.15.38:0/387690683 conn(0x7f5418079ba0 msgr2=0x7f5418079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 -- 172.21.15.38:0/387690683 shutdown_connections 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.726+0000 7f541e08b700 1 -- 172.21.15.38:0/387690683 wait complete. 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 Processor -- start 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 -- start start 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 0x7f5418199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 0x7f5418199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418108a50 0x7f541819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5418112d60 con 0x7f5418102a90 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5418112be0 con 0x7f5418108a50 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f541e08b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5418112ee0 con 0x7f5418103450 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f54177fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 0x7f5418199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f54177fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 0x7f5418199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58798/0 (socket says 172.21.15.38:58798) 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.727+0000 7f54177fe700 1 -- 172.21.15.38:0/324249068 learned_addr learned my addr 172.21.15.38:0/324249068 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5417fff700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418108a50 0x7f541819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5416ffd700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 0x7f5418199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f54177fe700 1 -- 172.21.15.38:0/324249068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 msgr2=0x7f5418199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f54177fe700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 0x7f5418199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f54177fe700 1 -- 172.21.15.38:0/324249068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418108a50 msgr2=0x7f541819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f54177fe700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418108a50 0x7f541819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f54177fe700 1 -- 172.21.15.38:0/324249068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541819e220 con 0x7f5418102a90 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5417fff700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418108a50 0x7f541819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5416ffd700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 0x7f5418199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f54177fe700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 0x7f5418199210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5408000c00 tx=0x7f5408004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408027070 con 0x7f5418102a90 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541819e4b0 con 0x7f5418102a90 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5408003fb0 con 0x7f5418102a90 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408022e60 con 0x7f5418102a90 2024-04-25T13:04:05.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.728+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54181a5d90 con 0x7f5418102a90 2024-04-25T13:04:05.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.729+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5418069730 con 0x7f5418102a90 2024-04-25T13:04:05.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.731+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f540802a070 con 0x7f5418102a90 2024-04-25T13:04:05.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.731+0000 7f5414ff9700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f54000a3cd0 0x7f54000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:05.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.732+0000 7f5416ffd700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f54000a3cd0 0x7f54000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:05.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.732+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f54080d1480 con 0x7f5418102a90 2024-04-25T13:04:05.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.732+0000 7f5416ffd700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f54000a3cd0 0x7f54000a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f540c005d90 tx=0x7f540c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.736+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5408013070 con 0x7f5418102a90 2024-04-25T13:04:05.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.939+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5418062e80 con 0x7f54000a3cd0 2024-04-25T13:04:05.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.946+0000 7f5414ff9700 1 -- 172.21.15.38:0/324249068 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259079 (secure 0 0 0) 0x7f5418062e80 con 0x7f54000a3cd0 2024-04-25T13:04:05.949 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f54000a3cd0 msgr2=0x7f54000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f54000a3cd0 0x7f54000a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f540c005d90 tx=0x7f540c005d00 comp rx=0 tx=0).stop 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 msgr2=0x7f5418199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 0x7f5418199210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5408000c00 tx=0x7f5408004580 comp rx=0 tx=0).stop 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 shutdown_connections 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f54000a3cd0 0x7f54000a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5418108a50 0x7f541819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5418102a90 0x7f5418199210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 --2- 172.21.15.38:0/324249068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5418103450 0x7f5418199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 >> 172.21.15.38:0/324249068 conn(0x7f5418079ba0 msgr2=0x7f54181018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 shutdown_connections 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:05.949+0000 7f541e08b700 1 -- 172.21.15.38:0/324249068 wait complete. 2024-04-25T13:04:05.952 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T13:04:06.693 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-04-25T13:04:05.302726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655936,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":902,"num_read_kb":577,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181033,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373496,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568328,"statfs":{"total":1151940427776,"available":1151557967872,"internally_reserved":0,"allocated":12079104,"data_stored":7473036,"data_compressed":185424,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002986"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120552+0000","last_change":"2024-04-25T13:02:44.736489+0000","last_active":"2024-04-25T13:03:37.120552+0000","last_peered":"2024-04-25T13:03:37.120552+0000","last_clean":"2024-04-25T13:03:37.120552+0000","last_became_active":"2024-04-25T13:02:44.735677+0000","last_became_peered":"2024-04-25T13:02:44.735677+0000","last_unstale":"2024-04-25T13:03:37.120552+0000","last_undegraded":"2024-04-25T13:03:37.120552+0000","last_fullsized":"2024-04-25T13:03:37.120552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:59.810878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.125057+0000","last_change":"2024-04-25T13:02:44.740349+0000","last_active":"2024-04-25T13:03:37.125057+0000","last_peered":"2024-04-25T13:03:37.125057+0000","last_clean":"2024-04-25T13:03:37.125057+0000","last_became_active":"2024-04-25T13:02:44.740024+0000","last_became_peered":"2024-04-25T13:02:44.740024+0000","last_unstale":"2024-04-25T13:03:37.125057+0000","last_undegraded":"2024-04-25T13:03:37.125057+0000","last_fullsized":"2024-04-25T13:03:37.125057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:35.763278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124578+0000","last_change":"2024-04-25T13:02:44.742136+0000","last_active":"2024-04-25T13:03:37.124578+0000","last_peered":"2024-04-25T13:03:37.124578+0000","last_clean":"2024-04-25T13:03:37.124578+0000","last_became_active":"2024-04-25T13:02:44.741840+0000","last_became_peered":"2024-04-25T13:02:44.741840+0000","last_unstale":"2024-04-25T13:03:37.124578+0000","last_undegraded":"2024-04-25T13:03:37.124578+0000","last_fullsized":"2024-04-25T13:03:37.124578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:48.690435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.210095+0000","last_change":"2024-04-25T13:02:44.733349+0000","last_active":"2024-04-25T13:03:37.210095+0000","last_peered":"2024-04-25T13:03:37.210095+0000","last_clean":"2024-04-25T13:03:37.210095+0000","last_became_active":"2024-04-25T13:02:44.732389+0000","last_became_peered":"2024-04-25T13:02:44.732389+0000","last_unstale":"2024-04-25T13:03:37.210095+0000","last_undegraded":"2024-04-25T13:03:37.210095+0000","last_fullsized":"2024-04-25T13:03:37.210095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:28.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124148+0000","last_change":"2024-04-25T13:02:44.732852+0000","last_active":"2024-04-25T13:03:37.124148+0000","last_peered":"2024-04-25T13:03:37.124148+0000","last_clean":"2024-04-25T13:03:37.124148+0000","last_became_active":"2024-04-25T13:02:44.732536+0000","last_became_peered":"2024-04-25T13:02:44.732536+0000","last_unstale":"2024-04-25T13:03:37.124148+0000","last_undegraded":"2024-04-25T13:03:37.124148+0000","last_fullsized":"2024-04-25T13:03:37.124148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:58.358684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.122892+0000","last_change":"2024-04-25T13:02:44.731636+0000","last_active":"2024-04-25T13:03:37.122892+0000","last_peered":"2024-04-25T13:03:37.122892+0000","last_clean":"2024-04-25T13:03:37.122892+0000","last_became_active":"2024-04-25T13:02:44.731334+0000","last_became_peered":"2024-04-25T13:02:44.731334+0000","last_unstale":"2024-04-25T13:03:37.122892+0000","last_undegraded":"2024-04-25T13:03:37.122892+0000","last_fullsized":"2024-04-25T13:03:37.122892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:00.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123336+0000","last_change":"2024-04-25T13:02:44.737275+0000","last_active":"2024-04-25T13:03:37.123336+0000","last_peered":"2024-04-25T13:03:37.123336+0000","last_clean":"2024-04-25T13:03:37.123336+0000","last_became_active":"2024-04-25T13:02:44.736924+0000","last_became_peered":"2024-04-25T13:02:44.736924+0000","last_unstale":"2024-04-25T13:03:37.123336+0000","last_undegraded":"2024-04-25T13:03:37.123336+0000","last_fullsized":"2024-04-25T13:03:37.123336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.564465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123131+0000","last_change":"2024-04-25T13:02:44.736736+0000","last_active":"2024-04-25T13:03:37.123131+0000","last_peered":"2024-04-25T13:03:37.123131+0000","last_clean":"2024-04-25T13:03:37.123131+0000","last_became_active":"2024-04-25T13:02:44.736403+0000","last_became_peered":"2024-04-25T13:02:44.736403+0000","last_unstale":"2024-04-25T13:03:37.123131+0000","last_undegraded":"2024-04-25T13:03:37.123131+0000","last_fullsized":"2024-04-25T13:03:37.123131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:54.531475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212262+0000","last_change":"2024-04-25T13:02:44.738124+0000","last_active":"2024-04-25T13:03:37.212262+0000","last_peered":"2024-04-25T13:03:37.212262+0000","last_clean":"2024-04-25T13:03:37.212262+0000","last_became_active":"2024-04-25T13:02:44.737572+0000","last_became_peered":"2024-04-25T13:02:44.737572+0000","last_unstale":"2024-04-25T13:03:37.212262+0000","last_undegraded":"2024-04-25T13:03:37.212262+0000","last_fullsized":"2024-04-25T13:03:37.212262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:00.692786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123141+0000","last_change":"2024-04-25T13:02:44.730108+0000","last_active":"2024-04-25T13:03:37.123141+0000","last_peered":"2024-04-25T13:03:37.123141+0000","last_clean":"2024-04-25T13:03:37.123141+0000","last_became_active":"2024-04-25T13:02:44.729449+0000","last_became_peered":"2024-04-25T13:02:44.729449+0000","last_unstale":"2024-04-25T13:03:37.123141+0000","last_undegraded":"2024-04-25T13:03:37.123141+0000","last_fullsized":"2024-04-25T13:03:37.123141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:19.888190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123345+0000","last_change":"2024-04-25T13:02:44.730902+0000","last_active":"2024-04-25T13:03:37.123345+0000","last_peered":"2024-04-25T13:03:37.123345+0000","last_clean":"2024-04-25T13:03:37.123345+0000","last_became_active":"2024-04-25T13:02:44.730560+0000","last_became_peered":"2024-04-25T13:02:44.730560+0000","last_unstale":"2024-04-25T13:03:37.123345+0000","last_undegraded":"2024-04-25T13:03:37.123345+0000","last_fullsized":"2024-04-25T13:03:37.123345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:07.133221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.210084+0000","last_change":"2024-04-25T13:02:44.730966+0000","last_active":"2024-04-25T13:03:37.210084+0000","last_peered":"2024-04-25T13:03:37.210084+0000","last_clean":"2024-04-25T13:03:37.210084+0000","last_became_active":"2024-04-25T13:02:44.730546+0000","last_became_peered":"2024-04-25T13:02:44.730546+0000","last_unstale":"2024-04-25T13:03:37.210084+0000","last_undegraded":"2024-04-25T13:03:37.210084+0000","last_fullsized":"2024-04-25T13:03:37.210084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.188491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212152+0000","last_change":"2024-04-25T13:02:44.739301+0000","last_active":"2024-04-25T13:03:37.212152+0000","last_peered":"2024-04-25T13:03:37.212152+0000","last_clean":"2024-04-25T13:03:37.212152+0000","last_became_active":"2024-04-25T13:02:44.738704+0000","last_became_peered":"2024-04-25T13:02:44.738704+0000","last_unstale":"2024-04-25T13:03:37.212152+0000","last_undegraded":"2024-04-25T13:03:37.212152+0000","last_fullsized":"2024-04-25T13:03:37.212152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:44.373522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123989+0000","last_change":"2024-04-25T13:02:44.738703+0000","last_active":"2024-04-25T13:03:37.123989+0000","last_peered":"2024-04-25T13:03:37.123989+0000","last_clean":"2024-04-25T13:03:37.123989+0000","last_became_active":"2024-04-25T13:02:44.738373+0000","last_became_peered":"2024-04-25T13:02:44.738373+0000","last_unstale":"2024-04-25T13:03:37.123989+0000","last_undegraded":"2024-04-25T13:03:37.123989+0000","last_fullsized":"2024-04-25T13:03:37.123989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:29.106615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124456+0000","last_change":"2024-04-25T13:02:44.743336+0000","last_active":"2024-04-25T13:03:37.124456+0000","last_peered":"2024-04-25T13:03:37.124456+0000","last_clean":"2024-04-25T13:03:37.124456+0000","last_became_active":"2024-04-25T13:02:44.743032+0000","last_became_peered":"2024-04-25T13:02:44.743032+0000","last_unstale":"2024-04-25T13:03:37.124456+0000","last_undegraded":"2024-04-25T13:03:37.124456+0000","last_fullsized":"2024-04-25T13:03:37.124456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:18.403138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211551+0000","last_change":"2024-04-25T13:02:44.739403+0000","last_active":"2024-04-25T13:03:37.211551+0000","last_peered":"2024-04-25T13:03:37.211551+0000","last_clean":"2024-04-25T13:03:37.211551+0000","last_became_active":"2024-04-25T13:02:44.739097+0000","last_became_peered":"2024-04-25T13:02:44.739097+0000","last_unstale":"2024-04-25T13:03:37.211551+0000","last_undegraded":"2024-04-25T13:03:37.211551+0000","last_fullsized":"2024-04-25T13:03:37.211551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:30.519343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120752+0000","last_change":"2024-04-25T13:02:44.736821+0000","last_active":"2024-04-25T13:03:37.120752+0000","last_peered":"2024-04-25T13:03:37.120752+0000","last_clean":"2024-04-25T13:03:37.120752+0000","last_became_active":"2024-04-25T13:02:44.736099+0000","last_became_peered":"2024-04-25T13:02:44.736099+0000","last_unstale":"2024-04-25T13:03:37.120752+0000","last_undegraded":"2024-04-25T13:03:37.120752+0000","last_fullsized":"2024-04-25T13:03:37.120752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:54.858845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126974+0000","last_change":"2024-04-25T13:02:44.731035+0000","last_active":"2024-04-25T13:03:37.126974+0000","last_peered":"2024-04-25T13:03:37.126974+0000","last_clean":"2024-04-25T13:03:37.126974+0000","last_became_active":"2024-04-25T13:02:44.730717+0000","last_became_peered":"2024-04-25T13:02:44.730717+0000","last_unstale":"2024-04-25T13:03:37.126974+0000","last_undegraded":"2024-04-25T13:03:37.126974+0000","last_fullsized":"2024-04-25T13:03:37.126974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:09.061183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123349+0000","last_change":"2024-04-25T13:02:44.735851+0000","last_active":"2024-04-25T13:03:37.123349+0000","last_peered":"2024-04-25T13:03:37.123349+0000","last_clean":"2024-04-25T13:03:37.123349+0000","last_became_active":"2024-04-25T13:02:44.735559+0000","last_became_peered":"2024-04-25T13:02:44.735559+0000","last_unstale":"2024-04-25T13:03:37.123349+0000","last_undegraded":"2024-04-25T13:03:37.123349+0000","last_fullsized":"2024-04-25T13:03:37.123349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.952094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:02.811312+0000","last_change":"2024-04-25T13:02:43.909800+0000","last_active":"2024-04-25T13:04:02.811312+0000","last_peered":"2024-04-25T13:04:02.811312+0000","last_clean":"2024-04-25T13:04:02.811312+0000","last_became_active":"2024-04-25T13:02:43.909463+0000","last_became_peered":"2024-04-25T13:02:43.909463+0000","last_unstale":"2024-04-25T13:04:02.811312+0000","last_undegraded":"2024-04-25T13:04:02.811312+0000","last_fullsized":"2024-04-25T13:04:02.811312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:31.495586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123734+0000","last_change":"2024-04-25T13:02:44.736710+0000","last_active":"2024-04-25T13:03:37.123734+0000","last_peered":"2024-04-25T13:03:37.123734+0000","last_clean":"2024-04-25T13:03:37.123734+0000","last_became_active":"2024-04-25T13:02:44.736385+0000","last_became_peered":"2024-04-25T13:02:44.736385+0000","last_unstale":"2024-04-25T13:03:37.123734+0000","last_undegraded":"2024-04-25T13:03:37.123734+0000","last_fullsized":"2024-04-25T13:03:37.123734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.216946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123889+0000","last_change":"2024-04-25T13:02:43.918766+0000","last_active":"2024-04-25T13:03:37.123889+0000","last_peered":"2024-04-25T13:03:37.123889+0000","last_clean":"2024-04-25T13:03:37.123889+0000","last_became_active":"2024-04-25T13:02:43.918283+0000","last_became_peered":"2024-04-25T13:02:43.918283+0000","last_unstale":"2024-04-25T13:03:37.123889+0000","last_undegraded":"2024-04-25T13:03:37.123889+0000","last_fullsized":"2024-04-25T13:03:37.123889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:17.986777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124830+0000","last_change":"2024-04-25T13:02:44.735806+0000","last_active":"2024-04-25T13:03:37.124830+0000","last_peered":"2024-04-25T13:03:37.124830+0000","last_clean":"2024-04-25T13:03:37.124830+0000","last_became_active":"2024-04-25T13:02:44.735507+0000","last_became_peered":"2024-04-25T13:02:44.735507+0000","last_unstale":"2024-04-25T13:03:37.124830+0000","last_undegraded":"2024-04-25T13:03:37.124830+0000","last_fullsized":"2024-04-25T13:03:37.124830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:22.230015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124588+0000","last_change":"2024-04-25T13:02:43.910055+0000","last_active":"2024-04-25T13:03:37.124588+0000","last_peered":"2024-04-25T13:03:37.124588+0000","last_clean":"2024-04-25T13:03:37.124588+0000","last_became_active":"2024-04-25T13:02:43.909762+0000","last_became_peered":"2024-04-25T13:02:43.909762+0000","last_unstale":"2024-04-25T13:03:37.124588+0000","last_undegraded":"2024-04-25T13:03:37.124588+0000","last_fullsized":"2024-04-25T13:03:37.124588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:47.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212007+0000","last_change":"2024-04-25T13:02:44.733253+0000","last_active":"2024-04-25T13:03:37.212007+0000","last_peered":"2024-04-25T13:03:37.212007+0000","last_clean":"2024-04-25T13:03:37.212007+0000","last_became_active":"2024-04-25T13:02:44.732917+0000","last_became_peered":"2024-04-25T13:02:44.732917+0000","last_unstale":"2024-04-25T13:03:37.212007+0000","last_undegraded":"2024-04-25T13:03:37.212007+0000","last_fullsized":"2024-04-25T13:03:37.212007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:10.380810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124091+0000","last_change":"2024-04-25T13:02:43.901490+0000","last_active":"2024-04-25T13:03:37.124091+0000","last_peered":"2024-04-25T13:03:37.124091+0000","last_clean":"2024-04-25T13:03:37.124091+0000","last_became_active":"2024-04-25T13:02:43.900807+0000","last_became_peered":"2024-04-25T13:02:43.900807+0000","last_unstale":"2024-04-25T13:03:37.124091+0000","last_undegraded":"2024-04-25T13:03:37.124091+0000","last_fullsized":"2024-04-25T13:03:37.124091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:48.101575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126748+0000","last_change":"2024-04-25T13:02:44.744380+0000","last_active":"2024-04-25T13:03:37.126748+0000","last_peered":"2024-04-25T13:03:37.126748+0000","last_clean":"2024-04-25T13:03:37.126748+0000","last_became_active":"2024-04-25T13:02:44.744098+0000","last_became_peered":"2024-04-25T13:02:44.744098+0000","last_unstale":"2024-04-25T13:03:37.126748+0000","last_undegraded":"2024-04-25T13:03:37.126748+0000","last_fullsized":"2024-04-25T13:03:37.126748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:53.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:00.802804+0000","last_change":"2024-04-25T13:02:43.908588+0000","last_active":"2024-04-25T13:04:00.802804+0000","last_peered":"2024-04-25T13:04:00.802804+0000","last_clean":"2024-04-25T13:04:00.802804+0000","last_became_active":"2024-04-25T13:02:43.908219+0000","last_became_peered":"2024-04-25T13:02:43.908219+0000","last_unstale":"2024-04-25T13:04:00.802804+0000","last_undegraded":"2024-04-25T13:04:00.802804+0000","last_fullsized":"2024-04-25T13:04:00.802804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:44.126509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123899+0000","last_change":"2024-04-25T13:02:44.737072+0000","last_active":"2024-04-25T13:03:37.123899+0000","last_peered":"2024-04-25T13:03:37.123899+0000","last_clean":"2024-04-25T13:03:37.123899+0000","last_became_active":"2024-04-25T13:02:44.736750+0000","last_became_peered":"2024-04-25T13:02:44.736750+0000","last_unstale":"2024-04-25T13:03:37.123899+0000","last_undegraded":"2024-04-25T13:03:37.123899+0000","last_fullsized":"2024-04-25T13:03:37.123899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:42.933740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123907+0000","last_change":"2024-04-25T13:02:43.913516+0000","last_active":"2024-04-25T13:03:37.123907+0000","last_peered":"2024-04-25T13:03:37.123907+0000","last_clean":"2024-04-25T13:03:37.123907+0000","last_became_active":"2024-04-25T13:02:43.913067+0000","last_became_peered":"2024-04-25T13:02:43.913067+0000","last_unstale":"2024-04-25T13:03:37.123907+0000","last_undegraded":"2024-04-25T13:03:37.123907+0000","last_fullsized":"2024-04-25T13:03:37.123907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:55.004598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126293+0000","last_change":"2024-04-25T13:02:44.741433+0000","last_active":"2024-04-25T13:03:37.126293+0000","last_peered":"2024-04-25T13:03:37.126293+0000","last_clean":"2024-04-25T13:03:37.126293+0000","last_became_active":"2024-04-25T13:02:44.741114+0000","last_became_peered":"2024-04-25T13:02:44.741114+0000","last_unstale":"2024-04-25T13:03:37.126293+0000","last_undegraded":"2024-04-25T13:03:37.126293+0000","last_fullsized":"2024-04-25T13:03:37.126293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:02.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211839+0000","last_change":"2024-04-25T13:02:43.900699+0000","last_active":"2024-04-25T13:03:37.211839+0000","last_peered":"2024-04-25T13:03:37.211839+0000","last_clean":"2024-04-25T13:03:37.211839+0000","last_became_active":"2024-04-25T13:02:43.900306+0000","last_became_peered":"2024-04-25T13:02:43.900306+0000","last_unstale":"2024-04-25T13:03:37.211839+0000","last_undegraded":"2024-04-25T13:03:37.211839+0000","last_fullsized":"2024-04-25T13:03:37.211839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.720365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123286+0000","last_change":"2024-04-25T13:02:44.733157+0000","last_active":"2024-04-25T13:03:37.123286+0000","last_peered":"2024-04-25T13:03:37.123286+0000","last_clean":"2024-04-25T13:03:37.123286+0000","last_became_active":"2024-04-25T13:02:44.732836+0000","last_became_peered":"2024-04-25T13:02:44.732836+0000","last_unstale":"2024-04-25T13:03:37.123286+0000","last_undegraded":"2024-04-25T13:03:37.123286+0000","last_fullsized":"2024-04-25T13:03:37.123286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:30.189708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126130+0000","last_change":"2024-04-25T13:02:43.913362+0000","last_active":"2024-04-25T13:03:37.126130+0000","last_peered":"2024-04-25T13:03:37.126130+0000","last_clean":"2024-04-25T13:03:37.126130+0000","last_became_active":"2024-04-25T13:02:43.912763+0000","last_became_peered":"2024-04-25T13:02:43.912763+0000","last_unstale":"2024-04-25T13:03:37.126130+0000","last_undegraded":"2024-04-25T13:03:37.126130+0000","last_fullsized":"2024-04-25T13:03:37.126130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:12.216165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126796+0000","last_change":"2024-04-25T13:02:44.732662+0000","last_active":"2024-04-25T13:03:37.126796+0000","last_peered":"2024-04-25T13:03:37.126796+0000","last_clean":"2024-04-25T13:03:37.126796+0000","last_became_active":"2024-04-25T13:02:44.732319+0000","last_became_peered":"2024-04-25T13:02:44.732319+0000","last_unstale":"2024-04-25T13:03:37.126796+0000","last_undegraded":"2024-04-25T13:03:37.126796+0000","last_fullsized":"2024-04-25T13:03:37.126796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:51.683916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":171,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:03.806700+0000","last_change":"2024-04-25T13:02:43.911019+0000","last_active":"2024-04-25T13:04:03.806700+0000","last_peered":"2024-04-25T13:04:03.806700+0000","last_clean":"2024-04-25T13:04:03.806700+0000","last_became_active":"2024-04-25T13:02:43.910707+0000","last_became_peered":"2024-04-25T13:02:43.910707+0000","last_unstale":"2024-04-25T13:04:03.806700+0000","last_undegraded":"2024-04-25T13:04:03.806700+0000","last_fullsized":"2024-04-25T13:04:03.806700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:07.042580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212101+0000","last_change":"2024-04-25T13:02:44.732791+0000","last_active":"2024-04-25T13:03:37.212101+0000","last_peered":"2024-04-25T13:03:37.212101+0000","last_clean":"2024-04-25T13:03:37.212101+0000","last_became_active":"2024-04-25T13:02:44.732484+0000","last_became_peered":"2024-04-25T13:02:44.732484+0000","last_unstale":"2024-04-25T13:03:37.212101+0000","last_undegraded":"2024-04-25T13:03:37.212101+0000","last_fullsized":"2024-04-25T13:03:37.212101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:36.164774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123437+0000","last_change":"2024-04-25T13:02:43.912447+0000","last_active":"2024-04-25T13:03:37.123437+0000","last_peered":"2024-04-25T13:03:37.123437+0000","last_clean":"2024-04-25T13:03:37.123437+0000","last_became_active":"2024-04-25T13:02:43.912091+0000","last_became_peered":"2024-04-25T13:02:43.912091+0000","last_unstale":"2024-04-25T13:03:37.123437+0000","last_undegraded":"2024-04-25T13:03:37.123437+0000","last_fullsized":"2024-04-25T13:03:37.123437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:41.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.125144+0000","last_change":"2024-04-25T13:02:44.739021+0000","last_active":"2024-04-25T13:03:37.125144+0000","last_peered":"2024-04-25T13:03:37.125144+0000","last_clean":"2024-04-25T13:03:37.125144+0000","last_became_active":"2024-04-25T13:02:44.737945+0000","last_became_peered":"2024-04-25T13:02:44.737945+0000","last_unstale":"2024-04-25T13:03:37.125144+0000","last_undegraded":"2024-04-25T13:03:37.125144+0000","last_fullsized":"2024-04-25T13:03:37.125144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:09.405021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126867+0000","last_change":"2024-04-25T13:02:43.909061+0000","last_active":"2024-04-25T13:03:37.126867+0000","last_peered":"2024-04-25T13:03:37.126867+0000","last_clean":"2024-04-25T13:03:37.126867+0000","last_became_active":"2024-04-25T13:02:43.908739+0000","last_became_peered":"2024-04-25T13:02:43.908739+0000","last_unstale":"2024-04-25T13:03:37.126867+0000","last_undegraded":"2024-04-25T13:03:37.126867+0000","last_fullsized":"2024-04-25T13:03:37.126867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:25.720808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212132+0000","last_change":"2024-04-25T13:02:44.734346+0000","last_active":"2024-04-25T13:03:37.212132+0000","last_peered":"2024-04-25T13:03:37.212132+0000","last_clean":"2024-04-25T13:03:37.212132+0000","last_became_active":"2024-04-25T13:02:44.734045+0000","last_became_peered":"2024-04-25T13:02:44.734045+0000","last_unstale":"2024-04-25T13:03:37.212132+0000","last_undegraded":"2024-04-25T13:03:37.212132+0000","last_fullsized":"2024-04-25T13:03:37.212132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:42.137455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:03.807147+0000","last_change":"2024-04-25T13:02:43.910212+0000","last_active":"2024-04-25T13:04:03.807147+0000","last_peered":"2024-04-25T13:04:03.807147+0000","last_clean":"2024-04-25T13:04:03.807147+0000","last_became_active":"2024-04-25T13:02:43.909873+0000","last_became_peered":"2024-04-25T13:02:43.909873+0000","last_unstale":"2024-04-25T13:04:03.807147+0000","last_undegraded":"2024-04-25T13:04:03.807147+0000","last_fullsized":"2024-04-25T13:04:03.807147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.716434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":134,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124406+0000","last_change":"2024-04-25T13:02:44.731633+0000","last_active":"2024-04-25T13:03:37.124406+0000","last_peered":"2024-04-25T13:03:37.124406+0000","last_clean":"2024-04-25T13:03:37.124406+0000","last_became_active":"2024-04-25T13:02:44.731360+0000","last_became_peered":"2024-04-25T13:02:44.731360+0000","last_unstale":"2024-04-25T13:03:37.124406+0000","last_undegraded":"2024-04-25T13:03:37.124406+0000","last_fullsized":"2024-04-25T13:03:37.124406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:12.960787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.137667+0000","last_change":"2024-04-25T13:03:37.137667+0000","last_active":"2024-04-25T13:03:37.137667+0000","last_peered":"2024-04-25T13:03:37.137667+0000","last_clean":"2024-04-25T13:03:37.137667+0000","last_became_active":"2024-04-25T13:03:37.125779+0000","last_became_peered":"2024-04-25T13:03:37.125779+0000","last_unstale":"2024-04-25T13:03:37.137667+0000","last_undegraded":"2024-04-25T13:03:37.137667+0000","last_fullsized":"2024-04-25T13:03:37.137667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:26.963757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212330+0000","last_change":"2024-04-25T13:02:44.742431+0000","last_active":"2024-04-25T13:03:37.212330+0000","last_peered":"2024-04-25T13:03:37.212330+0000","last_clean":"2024-04-25T13:03:37.212330+0000","last_became_active":"2024-04-25T13:02:44.742120+0000","last_became_peered":"2024-04-25T13:02:44.742120+0000","last_unstale":"2024-04-25T13:03:37.212330+0000","last_undegraded":"2024-04-25T13:03:37.212330+0000","last_fullsized":"2024-04-25T13:03:37.212330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:08.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126257+0000","last_change":"2024-04-25T13:02:43.911342+0000","last_active":"2024-04-25T13:03:37.126257+0000","last_peered":"2024-04-25T13:03:37.126257+0000","last_clean":"2024-04-25T13:03:37.126257+0000","last_became_active":"2024-04-25T13:02:43.910741+0000","last_became_peered":"2024-04-25T13:02:43.910741+0000","last_unstale":"2024-04-25T13:03:37.126257+0000","last_undegraded":"2024-04-25T13:03:37.126257+0000","last_fullsized":"2024-04-25T13:03:37.126257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:28.464712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123637+0000","last_change":"2024-04-25T13:02:44.736639+0000","last_active":"2024-04-25T13:03:37.123637+0000","last_peered":"2024-04-25T13:03:37.123637+0000","last_clean":"2024-04-25T13:03:37.123637+0000","last_became_active":"2024-04-25T13:02:44.736285+0000","last_became_peered":"2024-04-25T13:02:44.736285+0000","last_unstale":"2024-04-25T13:03:37.123637+0000","last_undegraded":"2024-04-25T13:03:37.123637+0000","last_fullsized":"2024-04-25T13:03:37.123637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:34.840298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:00.825537+0000","last_change":"2024-04-25T13:02:43.906893+0000","last_active":"2024-04-25T13:04:00.825537+0000","last_peered":"2024-04-25T13:04:00.825537+0000","last_clean":"2024-04-25T13:04:00.825537+0000","last_became_active":"2024-04-25T13:02:43.906538+0000","last_became_peered":"2024-04-25T13:02:43.906538+0000","last_unstale":"2024-04-25T13:04:00.825537+0000","last_undegraded":"2024-04-25T13:04:00.825537+0000","last_fullsized":"2024-04-25T13:04:00.825537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:10.601792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.122894+0000","last_change":"2024-04-25T13:02:44.741092+0000","last_active":"2024-04-25T13:03:37.122894+0000","last_peered":"2024-04-25T13:03:37.122894+0000","last_clean":"2024-04-25T13:03:37.122894+0000","last_became_active":"2024-04-25T13:02:44.740755+0000","last_became_peered":"2024-04-25T13:02:44.740755+0000","last_unstale":"2024-04-25T13:03:37.122894+0000","last_undegraded":"2024-04-25T13:03:37.122894+0000","last_fullsized":"2024-04-25T13:03:37.122894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:49.124211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123602+0000","last_change":"2024-04-25T13:02:43.909572+0000","last_active":"2024-04-25T13:03:37.123602+0000","last_peered":"2024-04-25T13:03:37.123602+0000","last_clean":"2024-04-25T13:03:37.123602+0000","last_became_active":"2024-04-25T13:02:43.909109+0000","last_became_peered":"2024-04-25T13:02:43.909109+0000","last_unstale":"2024-04-25T13:03:37.123602+0000","last_undegraded":"2024-04-25T13:03:37.123602+0000","last_fullsized":"2024-04-25T13:03:37.123602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.505599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126256+0000","last_change":"2024-04-25T13:02:44.740972+0000","last_active":"2024-04-25T13:03:37.126256+0000","last_peered":"2024-04-25T13:03:37.126256+0000","last_clean":"2024-04-25T13:03:37.126256+0000","last_became_active":"2024-04-25T13:02:44.740620+0000","last_became_peered":"2024-04-25T13:02:44.740620+0000","last_unstale":"2024-04-25T13:03:37.126256+0000","last_undegraded":"2024-04-25T13:03:37.126256+0000","last_fullsized":"2024-04-25T13:03:37.126256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:45.787763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124498+0000","last_change":"2024-04-25T13:02:43.901205+0000","last_active":"2024-04-25T13:03:37.124498+0000","last_peered":"2024-04-25T13:03:37.124498+0000","last_clean":"2024-04-25T13:03:37.124498+0000","last_became_active":"2024-04-25T13:02:43.900796+0000","last_became_peered":"2024-04-25T13:02:43.900796+0000","last_unstale":"2024-04-25T13:03:37.124498+0000","last_undegraded":"2024-04-25T13:03:37.124498+0000","last_fullsized":"2024-04-25T13:03:37.124498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:14.020387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211277+0000","last_change":"2024-04-25T13:03:37.211277+0000","last_active":"2024-04-25T13:03:37.211277+0000","last_peered":"2024-04-25T13:03:37.211277+0000","last_clean":"2024-04-25T13:03:37.211277+0000","last_became_active":"2024-04-25T13:03:37.210854+0000","last_became_peered":"2024-04-25T13:03:37.210854+0000","last_unstale":"2024-04-25T13:03:37.211277+0000","last_undegraded":"2024-04-25T13:03:37.211277+0000","last_fullsized":"2024-04-25T13:03:37.211277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.569591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126938+0000","last_change":"2024-04-25T13:02:43.911812+0000","last_active":"2024-04-25T13:03:37.126938+0000","last_peered":"2024-04-25T13:03:37.126938+0000","last_clean":"2024-04-25T13:03:37.126938+0000","last_became_active":"2024-04-25T13:02:43.911404+0000","last_became_peered":"2024-04-25T13:02:43.911404+0000","last_unstale":"2024-04-25T13:03:37.126938+0000","last_undegraded":"2024-04-25T13:03:37.126938+0000","last_fullsized":"2024-04-25T13:03:37.126938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:17.210980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123056+0000","last_change":"2024-04-25T13:02:44.732546+0000","last_active":"2024-04-25T13:03:37.123056+0000","last_peered":"2024-04-25T13:03:37.123056+0000","last_clean":"2024-04-25T13:03:37.123056+0000","last_became_active":"2024-04-25T13:02:44.732238+0000","last_became_peered":"2024-04-25T13:02:44.732238+0000","last_unstale":"2024-04-25T13:03:37.123056+0000","last_undegraded":"2024-04-25T13:03:37.123056+0000","last_fullsized":"2024-04-25T13:03:37.123056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.293939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:58.806376+0000","last_change":"2024-04-25T13:02:43.910620+0000","last_active":"2024-04-25T13:03:58.806376+0000","last_peered":"2024-04-25T13:03:58.806376+0000","last_clean":"2024-04-25T13:03:58.806376+0000","last_became_active":"2024-04-25T13:02:43.910325+0000","last_became_peered":"2024-04-25T13:02:43.910325+0000","last_unstale":"2024-04-25T13:03:58.806376+0000","last_undegraded":"2024-04-25T13:03:58.806376+0000","last_fullsized":"2024-04-25T13:03:58.806376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.185209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123028+0000","last_change":"2024-04-25T13:02:44.737134+0000","last_active":"2024-04-25T13:03:37.123028+0000","last_peered":"2024-04-25T13:03:37.123028+0000","last_clean":"2024-04-25T13:03:37.123028+0000","last_became_active":"2024-04-25T13:02:44.736790+0000","last_became_peered":"2024-04-25T13:02:44.736790+0000","last_unstale":"2024-04-25T13:03:37.123028+0000","last_undegraded":"2024-04-25T13:03:37.123028+0000","last_fullsized":"2024-04-25T13:03:37.123028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:02.680752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211874+0000","last_change":"2024-04-25T13:03:37.211039+0000","last_active":"2024-04-25T13:03:37.211874+0000","last_peered":"2024-04-25T13:03:37.211874+0000","last_clean":"2024-04-25T13:03:37.211874+0000","last_became_active":"2024-04-25T13:03:37.210634+0000","last_became_peered":"2024-04-25T13:03:37.210634+0000","last_unstale":"2024-04-25T13:03:37.211874+0000","last_undegraded":"2024-04-25T13:03:37.211874+0000","last_fullsized":"2024-04-25T13:03:37.211874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:43.772416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:02.827900+0000","last_change":"2024-04-25T13:02:43.920678+0000","last_active":"2024-04-25T13:04:02.827900+0000","last_peered":"2024-04-25T13:04:02.827900+0000","last_clean":"2024-04-25T13:04:02.827900+0000","last_became_active":"2024-04-25T13:02:43.920338+0000","last_became_peered":"2024-04-25T13:02:43.920338+0000","last_unstale":"2024-04-25T13:04:02.827900+0000","last_undegraded":"2024-04-25T13:04:02.827900+0000","last_fullsized":"2024-04-25T13:04:02.827900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123232+0000","last_change":"2024-04-25T13:02:44.736655+0000","last_active":"2024-04-25T13:03:37.123232+0000","last_peered":"2024-04-25T13:03:37.123232+0000","last_clean":"2024-04-25T13:03:37.123232+0000","last_became_active":"2024-04-25T13:02:44.736340+0000","last_became_peered":"2024-04-25T13:02:44.736340+0000","last_unstale":"2024-04-25T13:03:37.123232+0000","last_undegraded":"2024-04-25T13:03:37.123232+0000","last_fullsized":"2024-04-25T13:03:37.123232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:35.129606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.210381+0000","last_change":"2024-04-25T13:02:43.903179+0000","last_active":"2024-04-25T13:03:37.210381+0000","last_peered":"2024-04-25T13:03:37.210381+0000","last_clean":"2024-04-25T13:03:37.210381+0000","last_became_active":"2024-04-25T13:02:43.902799+0000","last_became_peered":"2024-04-25T13:02:43.902799+0000","last_unstale":"2024-04-25T13:03:37.210381+0000","last_undegraded":"2024-04-25T13:03:37.210381+0000","last_fullsized":"2024-04-25T13:03:37.210381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.472772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123636+0000","last_change":"2024-04-25T13:01:33.625244+0000","last_active":"2024-04-25T13:03:37.123636+0000","last_peered":"2024-04-25T13:03:37.123636+0000","last_clean":"2024-04-25T13:03:37.123636+0000","last_became_active":"2024-04-25T13:01:33.624760+0000","last_became_peered":"2024-04-25T13:01:33.624760+0000","last_unstale":"2024-04-25T13:03:37.123636+0000","last_undegraded":"2024-04-25T13:03:37.123636+0000","last_fullsized":"2024-04-25T13:03:37.123636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:58:01.939677+0000","last_clean_scrub_stamp":"2024-04-25T12:58:01.939677+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:21.636432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211698+0000","last_change":"2024-04-25T13:02:44.732711+0000","last_active":"2024-04-25T13:03:37.211698+0000","last_peered":"2024-04-25T13:03:37.211698+0000","last_clean":"2024-04-25T13:03:37.211698+0000","last_became_active":"2024-04-25T13:02:44.732394+0000","last_became_peered":"2024-04-25T13:02:44.732394+0000","last_unstale":"2024-04-25T13:03:37.211698+0000","last_undegraded":"2024-04-25T13:03:37.211698+0000","last_fullsized":"2024-04-25T13:03:37.211698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:02.601328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124195+0000","last_change":"2024-04-25T13:02:43.908773+0000","last_active":"2024-04-25T13:03:37.124195+0000","last_peered":"2024-04-25T13:03:37.124195+0000","last_clean":"2024-04-25T13:03:37.124195+0000","last_became_active":"2024-04-25T13:02:43.908440+0000","last_became_peered":"2024-04-25T13:02:43.908440+0000","last_unstale":"2024-04-25T13:03:37.124195+0000","last_undegraded":"2024-04-25T13:03:37.124195+0000","last_fullsized":"2024-04-25T13:03:37.124195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:49.565391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.143437+0000","last_change":"2024-04-25T13:03:37.143437+0000","last_active":"2024-04-25T13:03:37.143437+0000","last_peered":"2024-04-25T13:03:37.143437+0000","last_clean":"2024-04-25T13:03:37.143437+0000","last_became_active":"2024-04-25T13:03:37.127801+0000","last_became_peered":"2024-04-25T13:03:37.127801+0000","last_unstale":"2024-04-25T13:03:37.143437+0000","last_undegraded":"2024-04-25T13:03:37.143437+0000","last_fullsized":"2024-04-25T13:03:37.143437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:47.756708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212152+0000","last_change":"2024-04-25T13:02:44.730629+0000","last_active":"2024-04-25T13:03:37.212152+0000","last_peered":"2024-04-25T13:03:37.212152+0000","last_clean":"2024-04-25T13:03:37.212152+0000","last_became_active":"2024-04-25T13:02:44.730288+0000","last_became_peered":"2024-04-25T13:02:44.730288+0000","last_unstale":"2024-04-25T13:03:37.212152+0000","last_undegraded":"2024-04-25T13:03:37.212152+0000","last_fullsized":"2024-04-25T13:03:37.212152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:26.255183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.127606+0000","last_change":"2024-04-25T13:02:43.918791+0000","last_active":"2024-04-25T13:03:37.127606+0000","last_peered":"2024-04-25T13:03:37.127606+0000","last_clean":"2024-04-25T13:03:37.127606+0000","last_became_active":"2024-04-25T13:02:43.918454+0000","last_became_peered":"2024-04-25T13:02:43.918454+0000","last_unstale":"2024-04-25T13:03:37.127606+0000","last_undegraded":"2024-04-25T13:03:37.127606+0000","last_fullsized":"2024-04-25T13:03:37.127606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.137235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123965+0000","last_change":"2024-04-25T13:02:44.740324+0000","last_active":"2024-04-25T13:03:37.123965+0000","last_peered":"2024-04-25T13:03:37.123965+0000","last_clean":"2024-04-25T13:03:37.123965+0000","last_became_active":"2024-04-25T13:02:44.739993+0000","last_became_peered":"2024-04-25T13:02:44.739993+0000","last_unstale":"2024-04-25T13:03:37.123965+0000","last_undegraded":"2024-04-25T13:03:37.123965+0000","last_fullsized":"2024-04-25T13:03:37.123965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.207392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.209860+0000","last_change":"2024-04-25T13:02:43.903356+0000","last_active":"2024-04-25T13:03:37.209860+0000","last_peered":"2024-04-25T13:03:37.209860+0000","last_clean":"2024-04-25T13:03:37.209860+0000","last_became_active":"2024-04-25T13:02:43.903046+0000","last_became_peered":"2024-04-25T13:02:43.903046+0000","last_unstale":"2024-04-25T13:03:37.209860+0000","last_undegraded":"2024-04-25T13:03:37.209860+0000","last_fullsized":"2024-04-25T13:03:37.209860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.514373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.125168+0000","last_change":"2024-04-25T13:02:44.745033+0000","last_active":"2024-04-25T13:03:37.125168+0000","last_peered":"2024-04-25T13:03:37.125168+0000","last_clean":"2024-04-25T13:03:37.125168+0000","last_became_active":"2024-04-25T13:02:44.744718+0000","last_became_peered":"2024-04-25T13:02:44.744718+0000","last_unstale":"2024-04-25T13:03:37.125168+0000","last_undegraded":"2024-04-25T13:03:37.125168+0000","last_fullsized":"2024-04-25T13:03:37.125168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:26.736807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211810+0000","last_change":"2024-04-25T13:02:43.909366+0000","last_active":"2024-04-25T13:03:37.211810+0000","last_peered":"2024-04-25T13:03:37.211810+0000","last_clean":"2024-04-25T13:03:37.211810+0000","last_became_active":"2024-04-25T13:02:43.908968+0000","last_became_peered":"2024-04-25T13:02:43.908968+0000","last_unstale":"2024-04-25T13:03:37.211810+0000","last_undegraded":"2024-04-25T13:03:37.211810+0000","last_fullsized":"2024-04-25T13:03:37.211810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:36.915466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123714+0000","last_change":"2024-04-25T13:02:44.730481+0000","last_active":"2024-04-25T13:03:37.123714+0000","last_peered":"2024-04-25T13:03:37.123714+0000","last_clean":"2024-04-25T13:03:37.123714+0000","last_became_active":"2024-04-25T13:02:44.730163+0000","last_became_peered":"2024-04-25T13:02:44.730163+0000","last_unstale":"2024-04-25T13:03:37.123714+0000","last_undegraded":"2024-04-25T13:03:37.123714+0000","last_fullsized":"2024-04-25T13:03:37.123714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:12.207840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123352+0000","last_change":"2024-04-25T13:02:43.922262+0000","last_active":"2024-04-25T13:03:37.123352+0000","last_peered":"2024-04-25T13:03:37.123352+0000","last_clean":"2024-04-25T13:03:37.123352+0000","last_became_active":"2024-04-25T13:02:43.921356+0000","last_became_peered":"2024-04-25T13:02:43.921356+0000","last_unstale":"2024-04-25T13:03:37.123352+0000","last_undegraded":"2024-04-25T13:03:37.123352+0000","last_fullsized":"2024-04-25T13:03:37.123352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:27.446112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120335+0000","last_change":"2024-04-25T13:02:44.735879+0000","last_active":"2024-04-25T13:03:37.120335+0000","last_peered":"2024-04-25T13:03:37.120335+0000","last_clean":"2024-04-25T13:03:37.120335+0000","last_became_active":"2024-04-25T13:02:44.735574+0000","last_became_peered":"2024-04-25T13:02:44.735574+0000","last_unstale":"2024-04-25T13:03:37.120335+0000","last_undegraded":"2024-04-25T13:03:37.120335+0000","last_fullsized":"2024-04-25T13:03:37.120335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:14.094813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123231+0000","last_change":"2024-04-25T13:02:43.912250+0000","last_active":"2024-04-25T13:03:37.123231+0000","last_peered":"2024-04-25T13:03:37.123231+0000","last_clean":"2024-04-25T13:03:37.123231+0000","last_became_active":"2024-04-25T13:02:43.911945+0000","last_became_peered":"2024-04-25T13:02:43.911945+0000","last_unstale":"2024-04-25T13:03:37.123231+0000","last_undegraded":"2024-04-25T13:03:37.123231+0000","last_fullsized":"2024-04-25T13:03:37.123231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:51.872195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123175+0000","last_change":"2024-04-25T13:02:44.739184+0000","last_active":"2024-04-25T13:03:37.123175+0000","last_peered":"2024-04-25T13:03:37.123175+0000","last_clean":"2024-04-25T13:03:37.123175+0000","last_became_active":"2024-04-25T13:02:44.738862+0000","last_became_peered":"2024-04-25T13:02:44.738862+0000","last_unstale":"2024-04-25T13:03:37.123175+0000","last_undegraded":"2024-04-25T13:03:37.123175+0000","last_fullsized":"2024-04-25T13:03:37.123175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.605730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.215691+0000","last_change":"2024-04-25T13:03:37.215691+0000","last_active":"2024-04-25T13:03:37.215691+0000","last_peered":"2024-04-25T13:03:37.215691+0000","last_clean":"2024-04-25T13:03:37.215691+0000","last_became_active":"2024-04-25T13:03:37.211345+0000","last_became_peered":"2024-04-25T13:03:37.211345+0000","last_unstale":"2024-04-25T13:03:37.215691+0000","last_undegraded":"2024-04-25T13:03:37.215691+0000","last_fullsized":"2024-04-25T13:03:37.215691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:15.904620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124160+0000","last_change":"2024-04-25T13:02:44.738410+0000","last_active":"2024-04-25T13:03:37.124160+0000","last_peered":"2024-04-25T13:03:37.124160+0000","last_clean":"2024-04-25T13:03:37.124160+0000","last_became_active":"2024-04-25T13:02:44.738091+0000","last_became_peered":"2024-04-25T13:02:44.738091+0000","last_unstale":"2024-04-25T13:03:37.124160+0000","last_undegraded":"2024-04-25T13:03:37.124160+0000","last_fullsized":"2024-04-25T13:03:37.124160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.737907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:04:03.807458+0000","last_change":"2024-04-25T13:02:43.914563+0000","last_active":"2024-04-25T13:04:03.807458+0000","last_peered":"2024-04-25T13:04:03.807458+0000","last_clean":"2024-04-25T13:04:03.807458+0000","last_became_active":"2024-04-25T13:02:43.914274+0000","last_became_peered":"2024-04-25T13:02:43.914274+0000","last_unstale":"2024-04-25T13:04:03.807458+0000","last_undegraded":"2024-04-25T13:04:03.807458+0000","last_fullsized":"2024-04-25T13:04:03.807458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.124115+0000","last_change":"2024-04-25T13:02:44.730098+0000","last_active":"2024-04-25T13:03:37.124115+0000","last_peered":"2024-04-25T13:03:37.124115+0000","last_clean":"2024-04-25T13:03:37.124115+0000","last_became_active":"2024-04-25T13:02:44.729656+0000","last_became_peered":"2024-04-25T13:02:44.729656+0000","last_unstale":"2024-04-25T13:03:37.124115+0000","last_undegraded":"2024-04-25T13:03:37.124115+0000","last_fullsized":"2024-04-25T13:03:37.124115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.344967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126628+0000","last_change":"2024-04-25T13:02:43.911593+0000","last_active":"2024-04-25T13:03:37.126628+0000","last_peered":"2024-04-25T13:03:37.126628+0000","last_clean":"2024-04-25T13:03:37.126628+0000","last_became_active":"2024-04-25T13:02:43.910484+0000","last_became_peered":"2024-04-25T13:02:43.910484+0000","last_unstale":"2024-04-25T13:03:37.126628+0000","last_undegraded":"2024-04-25T13:03:37.126628+0000","last_fullsized":"2024-04-25T13:03:37.126628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:52.257269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.132602+0000","last_change":"2024-04-25T13:03:37.132602+0000","last_active":"2024-04-25T13:03:37.132602+0000","last_peered":"2024-04-25T13:03:37.132602+0000","last_clean":"2024-04-25T13:03:37.132602+0000","last_became_active":"2024-04-25T13:03:37.128042+0000","last_became_peered":"2024-04-25T13:03:37.128042+0000","last_unstale":"2024-04-25T13:03:37.132602+0000","last_undegraded":"2024-04-25T13:03:37.132602+0000","last_fullsized":"2024-04-25T13:03:37.132602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.577684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212472+0000","last_change":"2024-04-25T13:02:43.921129+0000","last_active":"2024-04-25T13:03:37.212472+0000","last_peered":"2024-04-25T13:03:37.212472+0000","last_clean":"2024-04-25T13:03:37.212472+0000","last_became_active":"2024-04-25T13:02:43.920768+0000","last_became_peered":"2024-04-25T13:02:43.920768+0000","last_unstale":"2024-04-25T13:03:37.212472+0000","last_undegraded":"2024-04-25T13:03:37.212472+0000","last_fullsized":"2024-04-25T13:03:37.212472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:42.872178+0000","last_clean_scrub_stamp":"2024-04-25T13:02:42.872178+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:49.178313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123113+0000","last_change":"2024-04-25T13:02:44.731934+0000","last_active":"2024-04-25T13:03:37.123113+0000","last_peered":"2024-04-25T13:03:37.123113+0000","last_clean":"2024-04-25T13:03:37.123113+0000","last_became_active":"2024-04-25T13:02:44.731624+0000","last_became_peered":"2024-04-25T13:02:44.731624+0000","last_unstale":"2024-04-25T13:03:37.123113+0000","last_undegraded":"2024-04-25T13:03:37.123113+0000","last_fullsized":"2024-04-25T13:03:37.123113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.809419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120309+0000","last_change":"2024-04-25T13:02:44.736982+0000","last_active":"2024-04-25T13:03:37.120309+0000","last_peered":"2024-04-25T13:03:37.120309+0000","last_clean":"2024-04-25T13:03:37.120309+0000","last_became_active":"2024-04-25T13:02:44.735127+0000","last_became_peered":"2024-04-25T13:02:44.735127+0000","last_unstale":"2024-04-25T13:03:37.120309+0000","last_undegraded":"2024-04-25T13:03:37.120309+0000","last_fullsized":"2024-04-25T13:03:37.120309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:06.827686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.213302+0000","last_change":"2024-04-25T13:03:37.213302+0000","last_active":"2024-04-25T13:03:37.213302+0000","last_peered":"2024-04-25T13:03:37.213302+0000","last_clean":"2024-04-25T13:03:37.213302+0000","last_became_active":"2024-04-25T13:03:37.212873+0000","last_became_peered":"2024-04-25T13:03:37.212873+0000","last_unstale":"2024-04-25T13:03:37.213302+0000","last_undegraded":"2024-04-25T13:03:37.213302+0000","last_fullsized":"2024-04-25T13:03:37.213302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:36.988761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123122+0000","last_change":"2024-04-25T13:02:44.739660+0000","last_active":"2024-04-25T13:03:37.123122+0000","last_peered":"2024-04-25T13:03:37.123122+0000","last_clean":"2024-04-25T13:03:37.123122+0000","last_became_active":"2024-04-25T13:02:44.738553+0000","last_became_peered":"2024-04-25T13:02:44.738553+0000","last_unstale":"2024-04-25T13:03:37.123122+0000","last_undegraded":"2024-04-25T13:03:37.123122+0000","last_fullsized":"2024-04-25T13:03:37.123122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:07.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.213046+0000","last_change":"2024-04-25T13:03:37.213046+0000","last_active":"2024-04-25T13:03:37.213046+0000","last_peered":"2024-04-25T13:03:37.213046+0000","last_clean":"2024-04-25T13:03:37.213046+0000","last_became_active":"2024-04-25T13:03:37.212649+0000","last_became_peered":"2024-04-25T13:03:37.212649+0000","last_unstale":"2024-04-25T13:03:37.213046+0000","last_undegraded":"2024-04-25T13:03:37.213046+0000","last_fullsized":"2024-04-25T13:03:37.213046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:57.193414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212110+0000","last_change":"2024-04-25T13:02:44.739324+0000","last_active":"2024-04-25T13:03:37.212110+0000","last_peered":"2024-04-25T13:03:37.212110+0000","last_clean":"2024-04-25T13:03:37.212110+0000","last_became_active":"2024-04-25T13:02:44.738994+0000","last_became_peered":"2024-04-25T13:02:44.738994+0000","last_unstale":"2024-04-25T13:03:37.212110+0000","last_undegraded":"2024-04-25T13:03:37.212110+0000","last_fullsized":"2024-04-25T13:03:37.212110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:50.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120646+0000","last_change":"2024-04-25T13:02:44.738075+0000","last_active":"2024-04-25T13:03:37.120646+0000","last_peered":"2024-04-25T13:03:37.120646+0000","last_clean":"2024-04-25T13:03:37.120646+0000","last_became_active":"2024-04-25T13:02:44.736616+0000","last_became_peered":"2024-04-25T13:02:44.736616+0000","last_unstale":"2024-04-25T13:03:37.120646+0000","last_undegraded":"2024-04-25T13:03:37.120646+0000","last_fullsized":"2024-04-25T13:03:37.120646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:14.875245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123380+0000","last_change":"2024-04-25T13:02:44.730926+0000","last_active":"2024-04-25T13:03:37.123380+0000","last_peered":"2024-04-25T13:03:37.123380+0000","last_clean":"2024-04-25T13:03:37.123380+0000","last_became_active":"2024-04-25T13:02:44.730154+0000","last_became_peered":"2024-04-25T13:02:44.730154+0000","last_unstale":"2024-04-25T13:03:37.123380+0000","last_undegraded":"2024-04-25T13:03:37.123380+0000","last_fullsized":"2024-04-25T13:03:37.123380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:07.361869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.128334+0000","last_change":"2024-04-25T13:03:37.128334+0000","last_active":"2024-04-25T13:03:37.128334+0000","last_peered":"2024-04-25T13:03:37.128334+0000","last_clean":"2024-04-25T13:03:37.128334+0000","last_became_active":"2024-04-25T13:03:37.127873+0000","last_became_peered":"2024-04-25T13:03:37.127873+0000","last_unstale":"2024-04-25T13:03:37.128334+0000","last_undegraded":"2024-04-25T13:03:37.128334+0000","last_fullsized":"2024-04-25T13:03:37.128334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.911113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.126280+0000","last_change":"2024-04-25T13:02:44.735555+0000","last_active":"2024-04-25T13:03:37.126280+0000","last_peered":"2024-04-25T13:03:37.126280+0000","last_clean":"2024-04-25T13:03:37.126280+0000","last_became_active":"2024-04-25T13:02:44.735238+0000","last_became_peered":"2024-04-25T13:02:44.735238+0000","last_unstale":"2024-04-25T13:03:37.126280+0000","last_undegraded":"2024-04-25T13:03:37.126280+0000","last_fullsized":"2024-04-25T13:03:37.126280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.611943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.123114+0000","last_change":"2024-04-25T13:02:44.730959+0000","last_active":"2024-04-25T13:03:37.123114+0000","last_peered":"2024-04-25T13:03:37.123114+0000","last_clean":"2024-04-25T13:03:37.123114+0000","last_became_active":"2024-04-25T13:02:44.730583+0000","last_became_peered":"2024-04-25T13:02:44.730583+0000","last_unstale":"2024-04-25T13:03:37.123114+0000","last_undegraded":"2024-04-25T13:03:37.123114+0000","last_fullsized":"2024-04-25T13:03:37.123114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.527837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.211809+0000","last_change":"2024-04-25T13:02:44.734174+0000","last_active":"2024-04-25T13:03:37.211809+0000","last_peered":"2024-04-25T13:03:37.211809+0000","last_clean":"2024-04-25T13:03:37.211809+0000","last_became_active":"2024-04-25T13:02:44.733785+0000","last_became_peered":"2024-04-25T13:02:44.733785+0000","last_unstale":"2024-04-25T13:03:37.211809+0000","last_undegraded":"2024-04-25T13:03:37.211809+0000","last_fullsized":"2024-04-25T13:03:37.211809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:39.641206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.212228+0000","last_change":"2024-04-25T13:02:44.740058+0000","last_active":"2024-04-25T13:03:37.212228+0000","last_peered":"2024-04-25T13:03:37.212228+0000","last_clean":"2024-04-25T13:03:37.212228+0000","last_became_active":"2024-04-25T13:02:44.739716+0000","last_became_peered":"2024-04-25T13:02:44.739716+0000","last_unstale":"2024-04-25T13:03:37.212228+0000","last_undegraded":"2024-04-25T13:03:37.212228+0000","last_fullsized":"2024-04-25T13:03:37.212228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:21.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:03:37.120271+0000","last_change":"2024-04-25T13:02:44.731335+0000","last_active":"2024-04-25T13:03:37.120271+0000","last_peered":"2024-04-25T13:03:37.120271+0000","last_clean":"2024-04-25T13:03:37.120271+0000","last_became_active":"2024-04-25T13:02:44.731017+0000","last_became_peered":"2024-04-25T13:02:44.731017+0000","last_unstale":"2024-04-25T13:03:37.120271+0000","last_undegraded":"2024-04-25T13:03:37.120271+0000","last_fullsized":"2024-04-25T13:03:37.120271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:02:43.879632+0000","last_clean_scrub_stamp":"2024-04-25T13:02:43.879632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:29.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":288217,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65334,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":766,"num_read_kb":458,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":196002,"data_compressed":5601,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":460141,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1109902,"data_compressed":44333,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":460008,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":460157,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1052396,"data_compressed":42466,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":459608,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":519433,"data_compressed":7692,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":460947,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":519585,"data_compressed":7692,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":459316,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459469,"data_compressed":5825,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":1052074,"data_compressed":42466,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35004,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36641,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60664,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60816,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60648,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:04:06.697 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:04:06.697 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:04:06.697 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:04:06.698 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:04:06.954 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:07.398 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:07 smithi038 ceph-mon[29400]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:07.398 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:07 smithi038 ceph-mon[29400]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:07.398 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:07 smithi038 ceph-mon[29400]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:04:07.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:07 smithi178 ceph-mon[33233]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:07.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:07 smithi178 ceph-mon[33233]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:07.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:07 smithi178 ceph-mon[33233]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:04:07.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:07 smithi104 ceph-mon[32726]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:07.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:07 smithi104 ceph-mon[32726]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:04:07.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:07 smithi104 ceph-mon[32726]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:04:08.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 -- 172.21.15.38:0/2188641806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40101960 msgr2=0x7fbc40101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 --2- 172.21.15.38:0/2188641806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40101960 0x7fbc40101de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009990 tx=0x7fbc34009ca0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 -- 172.21.15.38:0/2188641806 shutdown_connections 2024-04-25T13:04:08.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 --2- 172.21.15.38:0/2188641806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc40102320 0x7fbc4010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 --2- 172.21.15.38:0/2188641806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc40107960 0x7fbc40107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 --2- 172.21.15.38:0/2188641806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40101960 0x7fbc40101de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 -- 172.21.15.38:0/2188641806 >> 172.21.15.38:0/2188641806 conn(0x7fbc400789e0 msgr2=0x7fbc40078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 -- 172.21.15.38:0/2188641806 shutdown_connections 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.725+0000 7fbc4602f700 1 -- 172.21.15.38:0/2188641806 wait complete. 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 Processor -- start 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 -- start start 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc40101960 0x7fbc4019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc40102320 0x7fbc4019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 0x7fbc4019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40112db0 con 0x7fbc40107960 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40112c30 con 0x7fbc40101960 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc4602f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40112f30 con 0x7fbc40102320 2024-04-25T13:04:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc3ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 0x7fbc4019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc3ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 0x7fbc4019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58810/0 (socket says 172.21.15.38:58810) 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc3ffff700 1 -- 172.21.15.38:0/3656976005 learned_addr learned my addr 172.21.15.38:0/3656976005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc3effd700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc40102320 0x7fbc4019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.726+0000 7fbc3f7fe700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc40101960 0x7fbc4019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3ffff700 1 -- 172.21.15.38:0/3656976005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc40102320 msgr2=0x7fbc4019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3ffff700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc40102320 0x7fbc4019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3ffff700 1 -- 172.21.15.38:0/3656976005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc40101960 msgr2=0x7fbc4019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3ffff700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc40101960 0x7fbc4019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3ffff700 1 -- 172.21.15.38:0/3656976005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc401a03c0 con 0x7fbc40107960 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3ffff700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 0x7fbc4019fc80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc38007d70 tx=0x7fbc3800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38016020 con 0x7fbc40107960 2024-04-25T13:04:08.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbc3800ac90 con 0x7fbc40107960 2024-04-25T13:04:08.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc401a06b0 con 0x7fbc40107960 2024-04-25T13:04:08.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.727+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc380115e0 con 0x7fbc40107960 2024-04-25T13:04:08.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.728+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401a7f30 con 0x7fbc40107960 2024-04-25T13:04:08.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.728+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc40069730 con 0x7fbc40107960 2024-04-25T13:04:08.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.730+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbc38019020 con 0x7fbc40107960 2024-04-25T13:04:08.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.730+0000 7fbc3cff9700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbc280a3c40 0x7fbc280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:08.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.730+0000 7fbc3f7fe700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbc280a3c40 0x7fbc280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:08.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.730+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbc380d0330 con 0x7fbc40107960 2024-04-25T13:04:08.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.731+0000 7fbc3f7fe700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbc280a3c40 0x7fbc280a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005950 tx=0x7fbc3000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:08.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.735+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc38020020 con 0x7fbc40107960 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.992+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbc4019c710 con 0x7fbc40107960 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.993+0000 7fbc3cff9700 1 -- 172.21.15.38:0/3656976005 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbc4019c710 con 0x7fbc40107960 2024-04-25T13:04:08.991 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:08.992 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:04:08.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbc280a3c40 msgr2=0x7fbc280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbc280a3c40 0x7fbc280a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005950 tx=0x7fbc3000a300 comp rx=0 tx=0).stop 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 msgr2=0x7fbc4019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 0x7fbc4019fc80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc38007d70 tx=0x7fbc3800a340 comp rx=0 tx=0).stop 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 shutdown_connections 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbc280a3c40 0x7fbc280a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc40101960 0x7fbc4019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40107960 0x7fbc4019fc80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 --2- 172.21.15.38:0/3656976005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc40102320 0x7fbc4019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 >> 172.21.15.38:0/3656976005 conn(0x7fbc400789e0 msgr2=0x7fbc401042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 shutdown_connections 2024-04-25T13:04:08.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:08.995+0000 7fbc4602f700 1 -- 172.21.15.38:0/3656976005 wait complete. 2024-04-25T13:04:09.312 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:09 smithi038 ceph-mon[29400]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:09.312 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:09 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3656976005' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:04:09.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:09 smithi178 ceph-mon[33233]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:09.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:09 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3656976005' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:04:09.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:09 smithi104 ceph-mon[32726]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:09.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:09 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3656976005' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:04:09.684 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:04:09.684 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 -- 172.21.15.38:0/2269437469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 msgr2=0x7f8d2c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 --2- 172.21.15.38:0/2269437469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c1023c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 -- 172.21.15.38:0/2269437469 shutdown_connections 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 --2- 172.21.15.38:0/2269437469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103b50 0x7f8d2c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 --2- 172.21.15.38:0/2269437469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c103190 0x7f8d2c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 --2- 172.21.15.38:0/2269437469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c1023c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 -- 172.21.15.38:0/2269437469 >> 172.21.15.38:0/2269437469 conn(0x7f8d2c0fd530 msgr2=0x7f8d2c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 -- 172.21.15.38:0/2269437469 shutdown_connections 2024-04-25T13:04:09.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.791+0000 7f8d30915640 1 -- 172.21.15.38:0/2269437469 wait complete. 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 Processor -- start 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 -- start start 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103190 0x7f8d2c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c103b50 0x7f8d2c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c10dc00 con 0x7f8d2c101fa0 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c10da80 con 0x7f8d2c103190 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d30915640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c10dd80 con 0x7f8d2c103b50 2024-04-25T13:04:09.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40698/0 (socket says 172.21.15.38:40698) 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d2a59c640 1 -- 172.21.15.38:0/3026174934 learned_addr learned my addr 172.21.15.38:0/3026174934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.792+0000 7f8d29d9b640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103190 0x7f8d2c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d2a59c640 1 -- 172.21.15.38:0/3026174934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c103b50 msgr2=0x7f8d2c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d2a59c640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c103b50 0x7f8d2c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d2a59c640 1 -- 172.21.15.38:0/3026174934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103190 msgr2=0x7f8d2c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d2a59c640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103190 0x7f8d2c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d2a59c640 1 -- 172.21.15.38:0/3026174934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c19fe40 con 0x7f8d2c101fa0 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d29d9b640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103190 0x7f8d2c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d2a59c640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c198220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006010 tx=0x7f8d1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c03f030 con 0x7f8d2c101fa0 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c1a00d0 con 0x7f8d2c101fa0 2024-04-25T13:04:09.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8d1c003cd0 con 0x7f8d2c101fa0 2024-04-25T13:04:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035070 con 0x7f8d2c101fa0 2024-04-25T13:04:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.793+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a05c0 con 0x7f8d2c101fa0 2024-04-25T13:04:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.794+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c1092c0 con 0x7f8d2c101fa0 2024-04-25T13:04:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.795+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8d1c042030 con 0x7f8d2c101fa0 2024-04-25T13:04:09.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.796+0000 7f8d1b7fe640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8d140a3bb0 0x7f8d140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:09.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.796+0000 7f8d29d9b640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8d140a3bb0 0x7f8d140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:09.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.796+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8d1c0f0b60 con 0x7f8d2c101fa0 2024-04-25T13:04:09.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.796+0000 7f8d29d9b640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8d140a3bb0 0x7f8d140a6070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c199730 tx=0x7f8d2000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:09.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.801+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d1c0bd500 con 0x7f8d2c101fa0 2024-04-25T13:04:09.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.989+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d2c062b50 con 0x7f8d2c101fa0 2024-04-25T13:04:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.990+0000 7f8d1b7fe640 1 -- 172.21.15.38:0/3026174934 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22650 (secure 0 0 0) 0x7f8d1c0bce00 con 0x7f8d2c101fa0 2024-04-25T13:04:09.989 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:09.989 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:03:37.115660+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8d140a3bb0 msgr2=0x7f8d140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8d140a3bb0 0x7f8d140a6070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c199730 tx=0x7f8d2000a5c0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 msgr2=0x7f8d2c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c198220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006010 tx=0x7f8d1c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 shutdown_connections 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8d140a3bb0 0x7f8d140a6070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c103b50 0x7f8d2c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d2c103190 0x7f8d2c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 --2- 172.21.15.38:0/3026174934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d2c101fa0 0x7f8d2c198220 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 >> 172.21.15.38:0/3026174934 conn(0x7f8d2c0fd530 msgr2=0x7f8d2c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 shutdown_connections 2024-04-25T13:04:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:09.992+0000 7f8d30915640 1 -- 172.21.15.38:0/3026174934 wait complete. 2024-04-25T13:04:10.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:02:42.235731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:02:43.206179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:04:10.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:04:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 -- 172.21.15.38:0/2062109677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 msgr2=0x7f20e010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 --2- 172.21.15.38:0/2062109677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e010a3e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009ca0 tx=0x7f20d802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 -- 172.21.15.38:0/2062109677 shutdown_connections 2024-04-25T13:04:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 --2- 172.21.15.38:0/2062109677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e010a3e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 --2- 172.21.15.38:0/2062109677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20e0103190 0x7f20e0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 --2- 172.21.15.38:0/2062109677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 0x7f20e01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 -- 172.21.15.38:0/2062109677 >> 172.21.15.38:0/2062109677 conn(0x7f20e00fd530 msgr2=0x7f20e00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 -- 172.21.15.38:0/2062109677 shutdown_connections 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.110+0000 7f20e6f20640 1 -- 172.21.15.38:0/2062109677 wait complete. 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 Processor -- start 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 -- start start 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 0x7f20e01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20e0103190 0x7f20e0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20e010dd00 con 0x7f20e0103b50 2024-04-25T13:04:10.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20e010db80 con 0x7f20e0103190 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e6f20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20e010de80 con 0x7f20e0101fa0 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e019f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40720/0 (socket says 172.21.15.38:40720) 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 -- 172.21.15.38:0/1196150694 learned_addr learned my addr 172.21.15.38:0/1196150694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e4cbc640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 0x7f20e01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 -- 172.21.15.38:0/1196150694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 msgr2=0x7f20e01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 0x7f20e01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 -- 172.21.15.38:0/1196150694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20e0103190 msgr2=0x7f20e0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20e0103190 0x7f20e0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 -- 172.21.15.38:0/1196150694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e01a0010 con 0x7f20e0103b50 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e4cbc640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 0x7f20e01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20e54bd640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e019f9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009dc0 tx=0x7f20d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.111+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004240 con 0x7f20e0103b50 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.112+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f20d8031830 con 0x7f20e0103b50 2024-04-25T13:04:10.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.112+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e01a02a0 con 0x7f20e0103b50 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.112+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80305c0 con 0x7f20e0103b50 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.112+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e01a0870 con 0x7f20e0103b50 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.113+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e0109360 con 0x7f20e0103b50 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.114+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20d802f660 con 0x7f20e0103b50 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.115+0000 7f20ddffb640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f20d00a3bb0 0x7f20d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.115+0000 7f20e4cbc640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f20d00a3bb0 0x7f20d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.115+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f20d80f1eb0 con 0x7f20e0103b50 2024-04-25T13:04:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.115+0000 7f20e4cbc640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f20d00a3bb0 0x7f20d00a6070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20d4006fd0 tx=0x7f20d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.118+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20d80be890 con 0x7f20e0103b50 2024-04-25T13:04:10.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.304+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20e002d810 con 0x7f20e0103b50 2024-04-25T13:04:10.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.305+0000 7f20ddffb640 1 -- 172.21.15.38:0/1196150694 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f20d80be190 con 0x7f20e0103b50 2024-04-25T13:04:10.303 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.306+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f20d00a3bb0 msgr2=0x7f20d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.306+0000 7f20e6f20640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f20d00a3bb0 0x7f20d00a6070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20d4006fd0 tx=0x7f20d4008040 comp rx=0 tx=0).stop 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.306+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 msgr2=0x7f20e019f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.306+0000 7f20e6f20640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e019f9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009dc0 tx=0x7f20d8003c20 comp rx=0 tx=0).stop 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.306+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 shutdown_connections 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f20d00a3bb0 0x7f20d00a6070 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20e0103b50 0x7f20e019f9a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20e0103190 0x7f20e0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 --2- 172.21.15.38:0/1196150694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20e0101fa0 0x7f20e01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 >> 172.21.15.38:0/1196150694 conn(0x7f20e00fd530 msgr2=0x7f20e0106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 shutdown_connections 2024-04-25T13:04:10.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.307+0000 7f20e6f20640 1 -- 172.21.15.38:0/1196150694 wait complete. 2024-04-25T13:04:10.315 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:10 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3026174934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:10.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:04:10.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 -- 172.21.15.38:0/381500197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c103170 msgr2=0x7f044c1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 --2- 172.21.15.38:0/381500197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c103170 0x7f044c1035f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f043c009900 tx=0x7f043c02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:10.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 -- 172.21.15.38:0/381500197 shutdown_connections 2024-04-25T13:04:10.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 --2- 172.21.15.38:0/381500197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103b30 0x7f044c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 --2- 172.21.15.38:0/381500197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c103170 0x7f044c1035f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 --2- 172.21.15.38:0/381500197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c101f80 0x7f044c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 -- 172.21.15.38:0/381500197 >> 172.21.15.38:0/381500197 conn(0x7f044c0fd510 msgr2=0x7f044c0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 -- 172.21.15.38:0/381500197 shutdown_connections 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.423+0000 7f045378e640 1 -- 172.21.15.38:0/381500197 wait complete. 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 Processor -- start 2024-04-25T13:04:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 -- start start 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 0x7f044c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103170 0x7f044c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c103b30 0x7f044c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f044c10dc30 con 0x7f044c101f80 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f044c10dab0 con 0x7f044c103b30 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045378e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f044c10ddb0 con 0x7f044c103170 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 0x7f044c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 0x7f044c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40742/0 (socket says 172.21.15.38:40742) 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 -- 172.21.15.38:0/4044076119 learned_addr learned my addr 172.21.15.38:0/4044076119 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f0451d2b640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c103b30 0x7f044c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f0450d29640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103170 0x7f044c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 -- 172.21.15.38:0/4044076119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103170 msgr2=0x7f044c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103170 0x7f044c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 -- 172.21.15.38:0/4044076119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c103b30 msgr2=0x7f044c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c103b30 0x7f044c19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f045152a640 1 -- 172.21.15.38:0/4044076119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c19fea0 con 0x7f044c101f80 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.424+0000 7f0450d29640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103170 0x7f044c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.425+0000 7f045152a640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 0x7f044c198280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f044800d950 tx=0x7f044800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.425+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448017020 con 0x7f044c101f80 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.425+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0448009c30 con 0x7f044c101f80 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.425+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c1a0190 con 0x7f044c101f80 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.425+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800f740 con 0x7f044c101f80 2024-04-25T13:04:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.425+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c1a0700 con 0x7f044c101f80 2024-04-25T13:04:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.426+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044c1092a0 con 0x7f044c101f80 2024-04-25T13:04:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.427+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f044801a020 con 0x7f044c101f80 2024-04-25T13:04:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.428+0000 7f04427fc640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04380a3c30 0x7f04380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.428+0000 7f0450d29640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04380a3c30 0x7f04380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.428+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f04480d0010 con 0x7f044c101f80 2024-04-25T13:04:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.428+0000 7f0450d29640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04380a3c30 0x7f04380a60f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f043c000c00 tx=0x7f043c031530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.433+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0448009de0 con 0x7f044c101f80 2024-04-25T13:04:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:10 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3026174934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:10.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:10 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3026174934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:10.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.618+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f044c062b50 con 0x7f044c101f80 2024-04-25T13:04:10.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.618+0000 7f04427fc640 1 -- 172.21.15.38:0/4044076119 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f044809c9f0 con 0x7f044c101f80 2024-04-25T13:04:10.620 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T13:04:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04380a3c30 msgr2=0x7f04380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04380a3c30 0x7f04380a60f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f043c000c00 tx=0x7f043c031530 comp rx=0 tx=0).stop 2024-04-25T13:04:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 msgr2=0x7f044c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 0x7f044c198280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f044800d950 tx=0x7f044800de10 comp rx=0 tx=0).stop 2024-04-25T13:04:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 shutdown_connections 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04380a3c30 0x7f04380a60f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c103b30 0x7f044c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044c103170 0x7f044c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 --2- 172.21.15.38:0/4044076119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f044c101f80 0x7f044c198280 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 >> 172.21.15.38:0/4044076119 conn(0x7f044c0fd510 msgr2=0x7f044c106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 shutdown_connections 2024-04-25T13:04:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.620+0000 7f045378e640 1 -- 172.21.15.38:0/4044076119 wait complete. 2024-04-25T13:04:10.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 -- 172.21.15.38:0/2864317751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830102af0 msgr2=0x7fa830104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 --2- 172.21.15.38:0/2864317751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830102af0 0x7fa830104ee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009900 tx=0x7fa82c02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 -- 172.21.15.38:0/2864317751 shutdown_connections 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 --2- 172.21.15.38:0/2864317751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa830107d50 0x7fa83010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 --2- 172.21.15.38:0/2864317751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830105420 0x7fa830107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 --2- 172.21.15.38:0/2864317751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830102af0 0x7fa830104ee0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 -- 172.21.15.38:0/2864317751 >> 172.21.15.38:0/2864317751 conn(0x7fa8300fc420 msgr2=0x7fa8300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 -- 172.21.15.38:0/2864317751 shutdown_connections 2024-04-25T13:04:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 -- 172.21.15.38:0/2864317751 wait complete. 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.738+0000 7fa837f36640 1 Processor -- start 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 -- start start 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830102af0 0x7fa830102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa830105420 0x7fa830100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 0x7fa830100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa83010dc20 con 0x7fa830107d50 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa83010daa0 con 0x7fa830105420 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa837f36640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa83010dda0 con 0x7fa830102af0 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 0x7fa830100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 0x7fa830100fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40766/0 (socket says 172.21.15.38:40766) 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 -- 172.21.15.38:0/147028295 learned_addr learned my addr 172.21.15.38:0/147028295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa835cd2640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830102af0 0x7fa830102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8354d1640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa830105420 0x7fa830100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 -- 172.21.15.38:0/147028295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830102af0 msgr2=0x7fa830102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830102af0 0x7fa830102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 -- 172.21.15.38:0/147028295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa830105420 msgr2=0x7fa830100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa830105420 0x7fa830100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 -- 172.21.15.38:0/147028295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa830101870 con 0x7fa830107d50 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa8364d3640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 0x7fa830100fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa82800cce0 tx=0x7fa82800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.739+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82800f450 con 0x7fa830107d50 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.740+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa82800fac0 con 0x7fa830107d50 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.740+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8301a2360 con 0x7fa830107d50 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.740+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828018be0 con 0x7fa830107d50 2024-04-25T13:04:10.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.740+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8301a2860 con 0x7fa830107d50 2024-04-25T13:04:10.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.741+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8300fdf10 con 0x7fa830107d50 2024-04-25T13:04:10.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.742+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa82800fc70 con 0x7fa830107d50 2024-04-25T13:04:10.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.742+0000 7fa826ffd640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa81c0a3bb0 0x7fa81c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:10.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.742+0000 7fa835cd2640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa81c0a3bb0 0x7fa81c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:10.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.742+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa8280d0b40 con 0x7fa830107d50 2024-04-25T13:04:10.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.743+0000 7fa835cd2640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa81c0a3bb0 0x7fa81c0a6070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:10.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.747+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa828014030 con 0x7fa830107d50 2024-04-25T13:04:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.932+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa830062b50 con 0x7fa830107d50 2024-04-25T13:04:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.932+0000 7fa826ffd640 1 -- 172.21.15.38:0/147028295 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fa82809d520 con 0x7fa830107d50 2024-04-25T13:04:10.931 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa81c0a3bb0 msgr2=0x7fa81c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa81c0a3bb0 0x7fa81c0a6070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c031510 comp rx=0 tx=0).stop 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 msgr2=0x7fa830100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 0x7fa830100fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa82800cce0 tx=0x7fa82800a620 comp rx=0 tx=0).stop 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 shutdown_connections 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fa81c0a3bb0 0x7fa81c0a6070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa830107d50 0x7fa830100fe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa830105420 0x7fa830100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 --2- 172.21.15.38:0/147028295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830102af0 0x7fa830102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 >> 172.21.15.38:0/147028295 conn(0x7fa8300fc420 msgr2=0x7fa830103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:10.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 shutdown_connections 2024-04-25T13:04:10.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:10.934+0000 7fa837f36640 1 -- 172.21.15.38:0/147028295 wait complete. 2024-04-25T13:04:10.944 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 -- 172.21.15.38:0/3501465476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 msgr2=0x7f49e4107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/3501465476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e4107750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009900 tx=0x7f49d402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 -- 172.21.15.38:0/3501465476 shutdown_connections 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/3501465476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e4107c90 0x7f49e410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/3501465476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e4107750 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/3501465476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4102a30 0x7f49e4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 -- 172.21.15.38:0/3501465476 >> 172.21.15.38:0/3501465476 conn(0x7f49e40fc440 msgr2=0x7f49e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 -- 172.21.15.38:0/3501465476 shutdown_connections 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.047+0000 7f49ea0c2640 1 -- 172.21.15.38:0/3501465476 wait complete. 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 Processor -- start 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 -- start start 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e4102a30 0x7f49e41023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e41009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4107c90 0x7f49e4100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f49e410dc70 con 0x7f49e4105360 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49e410daf0 con 0x7f49e4107c90 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49ea0c2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49e410ddf0 con 0x7f49e4102a30 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e41009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e41009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40784/0 (socket says 172.21.15.38:40784) 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 -- 172.21.15.38:0/2416658051 learned_addr learned my addr 172.21.15.38:0/2416658051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:11.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e3fff640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4107c90 0x7f49e4100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e37fe640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e4102a30 0x7f49e41023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 -- 172.21.15.38:0/2416658051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e4102a30 msgr2=0x7f49e41023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e4102a30 0x7f49e41023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 -- 172.21.15.38:0/2416658051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4107c90 msgr2=0x7f49e4100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4107c90 0x7f49e4100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e2ffd640 1 -- 172.21.15.38:0/2416658051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e41017b0 con 0x7f49e4105360 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.048+0000 7f49e3fff640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4107c90 0x7f49e4100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.049+0000 7f49e2ffd640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e41009e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006010 tx=0x7f49d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.049+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d403f030 con 0x7f49e4105360 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.049+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f49d4003cd0 con 0x7f49e4105360 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.049+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e41a2400 con 0x7f49e4105360 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.049+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4035070 con 0x7f49e4105360 2024-04-25T13:04:11.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.049+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e41a29a0 con 0x7f49e4105360 2024-04-25T13:04:11.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.050+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e40fdf30 con 0x7f49e4105360 2024-04-25T13:04:11.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.051+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49d4042030 con 0x7f49e4105360 2024-04-25T13:04:11.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.052+0000 7f49e0ff9640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49cc0a3c30 0x7f49cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.052+0000 7f49e37fe640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49cc0a3c30 0x7f49cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.052+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49d40f0ab0 con 0x7f49e4105360 2024-04-25T13:04:11.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.053+0000 7f49e37fe640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49cc0a3c30 0x7f49cc0a60f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005fd0 tx=0x7f49d8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:11.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.057+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49d40bd490 con 0x7f49e4105360 2024-04-25T13:04:11.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.284+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49e402d5c0 con 0x7f49e4105360 2024-04-25T13:04:11.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.285+0000 7f49e0ff9640 1 -- 172.21.15.38:0/2416658051 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11938 (secure 0 0 0) 0x7f49d40bcd90 con 0x7f49e4105360 2024-04-25T13:04:11.283 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:11.283 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:02:51.699933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24484,24518,24482,24466,14625]},"id":1}]} 2024-04-25T13:04:11.284 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49cc0a3c30 msgr2=0x7f49cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49cc0a3c30 0x7f49cc0a60f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005fd0 tx=0x7f49d8005c70 comp rx=0 tx=0).stop 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 msgr2=0x7f49e41009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e41009e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006010 tx=0x7f49d40042f0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 shutdown_connections 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f49cc0a3c30 0x7f49cc0a60f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49e4107c90 0x7f49e4100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e4105360 0x7f49e41009e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 --2- 172.21.15.38:0/2416658051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e4102a30 0x7f49e41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 >> 172.21.15.38:0/2416658051 conn(0x7f49e40fc440 msgr2=0x7f49e4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 shutdown_connections 2024-04-25T13:04:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.287+0000 7f49ea0c2640 1 -- 172.21.15.38:0/2416658051 wait complete. 2024-04-25T13:04:11.297 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:11 smithi038 ceph-mon[29400]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:11.297 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:11 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1196150694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.297 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:11 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/4044076119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.297 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:11 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/147028295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:04:11.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 -- 172.21.15.38:0/623870167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 msgr2=0x7f372410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 --2- 172.21.15.38:0/623870167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372410a100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f371c009ca0 tx=0x7f371c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 -- 172.21.15.38:0/623870167 shutdown_connections 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 --2- 172.21.15.38:0/623870167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372410a100 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 --2- 172.21.15.38:0/623870167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37241053e0 0x7f37241077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 --2- 172.21.15.38:0/623870167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3724102ab0 0x7f3724104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 -- 172.21.15.38:0/623870167 >> 172.21.15.38:0/623870167 conn(0x7f37240fc440 msgr2=0x7f37240fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 -- 172.21.15.38:0/623870167 shutdown_connections 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 -- 172.21.15.38:0/623870167 wait complete. 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 Processor -- start 2024-04-25T13:04:11.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.402+0000 7f372a74f640 1 -- start start 2024-04-25T13:04:11.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3724102ab0 0x7f3724198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37241053e0 0x7f3724198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f372410dc50 con 0x7f3724107d10 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f372410dad0 con 0x7f3724102ab0 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f372410ddd0 con 0x7f37241053e0 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40798/0 (socket says 172.21.15.38:40798) 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 -- 172.21.15.38:0/3840342578 learned_addr learned my addr 172.21.15.38:0/3840342578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3723fff640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3724102ab0 0x7f3724198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f37237fe640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37241053e0 0x7f3724198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 -- 172.21.15.38:0/3840342578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37241053e0 msgr2=0x7f3724198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37241053e0 0x7f3724198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 -- 172.21.15.38:0/3840342578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3724102ab0 msgr2=0x7f3724198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3724102ab0 0x7f3724198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 -- 172.21.15.38:0/3840342578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372419fee0 con 0x7f3724107d10 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3723fff640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3724102ab0 0x7f3724198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f3728cec640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372419f870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f371c006010 tx=0x7f371c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c003bf0 con 0x7f3724107d10 2024-04-25T13:04:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f371c003d90 con 0x7f3724107d10 2024-04-25T13:04:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37241a0170 con 0x7f3724107d10 2024-04-25T13:04:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c02fc60 con 0x7f3724107d10 2024-04-25T13:04:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.403+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241a06c0 con 0x7f3724107d10 2024-04-25T13:04:11.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.405+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37240fdf30 con 0x7f3724107d10 2024-04-25T13:04:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.406+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f371c004040 con 0x7f3724107d10 2024-04-25T13:04:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.406+0000 7f37217fa640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f37100a3bb0 0x7f37100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.406+0000 7f3723fff640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f37100a3bb0 0x7f37100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.406+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f371c0f1c20 con 0x7f3724107d10 2024-04-25T13:04:11.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.407+0000 7f3723fff640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f37100a3bb0 0x7f37100a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f370c0058c0 tx=0x7f370c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.411+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f371c0be720 con 0x7f3724107d10 2024-04-25T13:04:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:11 smithi178 ceph-mon[33233]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:11 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1196150694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:11 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/4044076119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:11 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/147028295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:11 smithi104 ceph-mon[32726]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:11.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:11 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1196150694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:11 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/4044076119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:11 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/147028295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:11.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.600+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3724062b50 con 0x7f3724107d10 2024-04-25T13:04:11.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.601+0000 7f37217fa640 1 -- 172.21.15.38:0/3840342578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22650 (secure 0 0 0) 0x7f371c0be020 con 0x7f3724107d10 2024-04-25T13:04:11.600 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:11.601 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:03:37.115660+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.603+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f37100a3bb0 msgr2=0x7f37100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.603+0000 7f372a74f640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f37100a3bb0 0x7f37100a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f370c0058c0 tx=0x7f370c005850 comp rx=0 tx=0).stop 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.603+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 msgr2=0x7f372419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.603+0000 7f372a74f640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372419f870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f371c006010 tx=0x7f371c02f380 comp rx=0 tx=0).stop 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 shutdown_connections 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f37100a3bb0 0x7f37100a6070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3724107d10 0x7f372419f870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37241053e0 0x7f3724198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 --2- 172.21.15.38:0/3840342578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3724102ab0 0x7f3724198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 >> 172.21.15.38:0/3840342578 conn(0x7f37240fc440 msgr2=0x7f3724103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:11.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 shutdown_connections 2024-04-25T13:04:11.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.604+0000 7f372a74f640 1 -- 172.21.15.38:0/3840342578 wait complete. 2024-04-25T13:04:11.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:02:42.235731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:02:43.206179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:04:11.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:04:11.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.720+0000 7f1cd0607640 1 -- 172.21.15.38:0/1162650600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 msgr2=0x7f1cc8107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.720+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1162650600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc8107730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009900 tx=0x7f1cbc02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:11.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- 172.21.15.38:0/1162650600 shutdown_connections 2024-04-25T13:04:11.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1162650600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cc8107c70 0x7f1cc810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1162650600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc8107730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1162650600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8102a10 0x7f1cc8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- 172.21.15.38:0/1162650600 >> 172.21.15.38:0/1162650600 conn(0x7f1cc80fc400 msgr2=0x7f1cc80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- 172.21.15.38:0/1162650600 shutdown_connections 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- 172.21.15.38:0/1162650600 wait complete. 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 Processor -- start 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- start start 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cc8102a10 0x7f1cc819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8107c70 0x7f1cc81a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810dc00 con 0x7f1cc8105340 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810da80 con 0x7f1cc8102a10 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cd0607640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810dd80 con 0x7f1cc8107c70 2024-04-25T13:04:11.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1ccdba2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1ccdba2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40816/0 (socket says 172.21.15.38:40816) 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1ccdba2640 1 -- 172.21.15.38:0/1507615243 learned_addr learned my addr 172.21.15.38:0/1507615243 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.721+0000 7f1cce3a3640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cc8102a10 0x7f1cc819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cceba4640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8107c70 0x7f1cc81a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1ccdba2640 1 -- 172.21.15.38:0/1507615243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8107c70 msgr2=0x7f1cc81a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1ccdba2640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8107c70 0x7f1cc81a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1ccdba2640 1 -- 172.21.15.38:0/1507615243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cc8102a10 msgr2=0x7f1cc819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1ccdba2640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cc8102a10 0x7f1cc819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1ccdba2640 1 -- 172.21.15.38:0/1507615243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc81a2110 con 0x7f1cc8105340 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cceba4640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8107c70 0x7f1cc81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1ccdba2640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc819aa30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc006010 tx=0x7f1cbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc03f030 con 0x7f1cc8105340 2024-04-25T13:04:11.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1cbc003cd0 con 0x7f1cc8105340 2024-04-25T13:04:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc81a23a0 con 0x7f1cc8105340 2024-04-25T13:04:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc035070 con 0x7f1cc8105340 2024-04-25T13:04:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.722+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc81a2890 con 0x7f1cc8105340 2024-04-25T13:04:11.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.723+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc81097c0 con 0x7f1cc8105340 2024-04-25T13:04:11.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.724+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1cbc042030 con 0x7f1cc8105340 2024-04-25T13:04:11.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.725+0000 7f1cc77fe640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1cb00a3bb0 0x7f1cb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:11.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.725+0000 7f1cce3a3640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1cb00a3bb0 0x7f1cb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:11.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.725+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1cbc0f09b0 con 0x7f1cc8105340 2024-04-25T13:04:11.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.725+0000 7f1cce3a3640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1cb00a3bb0 0x7f1cb00a6070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0005fd0 tx=0x7f1cc000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:11.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.730+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cbc0bd350 con 0x7f1cc8105340 2024-04-25T13:04:11.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.917+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cc8062b50 con 0x7f1cc8105340 2024-04-25T13:04:11.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.918+0000 7f1cc77fe640 1 -- 172.21.15.38:0/1507615243 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f1cbc0bcc50 con 0x7f1cc8105340 2024-04-25T13:04:11.917 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1cb00a3bb0 msgr2=0x7f1cb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1cb00a3bb0 0x7f1cb00a6070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0005fd0 tx=0x7f1cc000b390 comp rx=0 tx=0).stop 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 msgr2=0x7f1cc819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc819aa30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc006010 tx=0x7f1cbc0042f0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 shutdown_connections 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1cb00a3bb0 0x7f1cb00a6070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cc8107c70 0x7f1cc81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cc8105340 0x7f1cc819aa30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 --2- 172.21.15.38:0/1507615243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cc8102a10 0x7f1cc819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 >> 172.21.15.38:0/1507615243 conn(0x7f1cc80fc400 msgr2=0x7f1cc81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 shutdown_connections 2024-04-25T13:04:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:11.920+0000 7f1cd0607640 1 -- 172.21.15.38:0/1507615243 wait complete. 2024-04-25T13:04:11.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:04:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 -- 172.21.15.38:0/1083548821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08101f80 msgr2=0x7f8b081023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1083548821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08101f80 0x7f8b081023a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009900 tx=0x7f8af802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 -- 172.21.15.38:0/1083548821 shutdown_connections 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1083548821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b08103b30 0x7f8b0810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1083548821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b08103170 0x7f8b081035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1083548821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08101f80 0x7f8b081023a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 -- 172.21.15.38:0/1083548821 >> 172.21.15.38:0/1083548821 conn(0x7f8b080fd510 msgr2=0x7f8b080ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 -- 172.21.15.38:0/1083548821 shutdown_connections 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.037+0000 7f8b0e337640 1 -- 172.21.15.38:0/1083548821 wait complete. 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 Processor -- start 2024-04-25T13:04:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 -- start start 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b08101f80 0x7f8b08198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 0x7f8b08198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b08103b30 0x7f8b0819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810dbe0 con 0x7f8b08103170 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810da60 con 0x7f8b08103b30 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0e337640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810dd60 con 0x7f8b08101f80 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 0x7f8b08198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 0x7f8b08198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40826/0 (socket says 172.21.15.38:40826) 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 -- 172.21.15.38:0/1932410101 learned_addr learned my addr 172.21.15.38:0/1932410101 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b07fff640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b08101f80 0x7f8b08198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b0c8d4640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b08103b30 0x7f8b0819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 -- 172.21.15.38:0/1932410101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b08101f80 msgr2=0x7f8b08198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b08101f80 0x7f8b08198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 -- 172.21.15.38:0/1932410101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b08103b30 msgr2=0x7f8b0819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b08103b30 0x7f8b0819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 -- 172.21.15.38:0/1932410101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0819fe40 con 0x7f8b08103170 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.038+0000 7f8b077fe640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 0x7f8b08198760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c920 tx=0x7f8afc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.039+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc007c70 con 0x7f8b08103170 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.039+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8afc007e10 con 0x7f8b08103170 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.039+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b081a0130 con 0x7f8b08103170 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.039+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0188c0 con 0x7f8b08103170 2024-04-25T13:04:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.039+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b081a0680 con 0x7f8b08103170 2024-04-25T13:04:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.040+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b08109240 con 0x7f8b08103170 2024-04-25T13:04:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.041+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8afc018a60 con 0x7f8b08103170 2024-04-25T13:04:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.041+0000 7f8b057fa640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8af00a3bb0 0x7f8af00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.042+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8afc0d05f0 con 0x7f8b08103170 2024-04-25T13:04:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.042+0000 7f8b07fff640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8af00a3bb0 0x7f8af00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.042+0000 7f8b07fff640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8af00a3bb0 0x7f8af00a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8af8000c00 tx=0x7f8af8031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.046+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8afc014030 con 0x7f8b08103170 2024-04-25T13:04:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.234+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b08062b50 con 0x7f8b08103170 2024-04-25T13:04:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.235+0000 7f8b057fa640 1 -- 172.21.15.38:0/1932410101 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f8afc00de20 con 0x7f8b08103170 2024-04-25T13:04:12.233 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T13:04:12.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8af00a3bb0 msgr2=0x7f8af00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8af00a3bb0 0x7f8af00a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8af8000c00 tx=0x7f8af8031510 comp rx=0 tx=0).stop 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 msgr2=0x7f8b08198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 0x7f8b08198760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c920 tx=0x7f8afc00cde0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 shutdown_connections 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8af00a3bb0 0x7f8af00a6070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b08103b30 0x7f8b0819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b08103170 0x7f8b08198760 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 --2- 172.21.15.38:0/1932410101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b08101f80 0x7f8b08198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 >> 172.21.15.38:0/1932410101 conn(0x7f8b080fd510 msgr2=0x7f8b08106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 shutdown_connections 2024-04-25T13:04:12.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.237+0000 7f8b0e337640 1 -- 172.21.15.38:0/1932410101 wait complete. 2024-04-25T13:04:12.245 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:12 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2416658051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:12.245 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:12 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3840342578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:12.245 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:12 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1507615243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:12.247 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 -- 172.21.15.38:0/3906321831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb764101fe0 msgr2=0x7fb764102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 --2- 172.21.15.38:0/3906321831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb764101fe0 0x7fb764102460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb754009900 tx=0x7fb75402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 -- 172.21.15.38:0/3906321831 shutdown_connections 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 --2- 172.21.15.38:0/3906321831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7641029a0 0x7fb7641092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 --2- 172.21.15.38:0/3906321831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb764101fe0 0x7fb764102460 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 --2- 172.21.15.38:0/3906321831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764100e90 0x7fb7641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 -- 172.21.15.38:0/3906321831 >> 172.21.15.38:0/3906321831 conn(0x7fb7640fc400 msgr2=0x7fb7640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 -- 172.21.15.38:0/3906321831 shutdown_connections 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.353+0000 7fb76b936640 1 -- 172.21.15.38:0/3906321831 wait complete. 2024-04-25T13:04:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 Processor -- start 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 -- start start 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb764100e90 0x7fb764069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764101fe0 0x7fb76419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 0x7fb76419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb76410cb10 con 0x7fb7641029a0 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb76410c990 con 0x7fb764101fe0 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb76b936640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb76410cc90 con 0x7fb764100e90 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 0x7fb76419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 0x7fb76419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40838/0 (socket says 172.21.15.38:40838) 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 -- 172.21.15.38:0/458804788 learned_addr learned my addr 172.21.15.38:0/458804788 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb7696d2640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb764100e90 0x7fb764069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb768ed1640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764101fe0 0x7fb76419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 -- 172.21.15.38:0/458804788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb764100e90 msgr2=0x7fb764069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb764100e90 0x7fb764069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 -- 172.21.15.38:0/458804788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764101fe0 msgr2=0x7fb76419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764101fe0 0x7fb76419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb769ed3640 1 -- 172.21.15.38:0/458804788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76419ff60 con 0x7fb7641029a0 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb768ed1640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764101fe0 0x7fb76419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.354+0000 7fb7696d2640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb764100e90 0x7fb764069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.355+0000 7fb769ed3640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 0x7fb76419f650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00d750 tx=0x7fb75c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.355+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c017070 con 0x7fb7641029a0 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.355+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb75c0054f0 con 0x7fb7641029a0 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.355+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7641a01f0 con 0x7fb7641029a0 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.355+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c004cd0 con 0x7fb7641029a0 2024-04-25T13:04:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.355+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641a06f0 con 0x7fb7641029a0 2024-04-25T13:04:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.356+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764108270 con 0x7fb7641029a0 2024-04-25T13:04:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.357+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb75c009bd0 con 0x7fb7641029a0 2024-04-25T13:04:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.358+0000 7fb75a7fc640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.358+0000 7fb7696d2640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.358+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb75c0cff40 con 0x7fb7641029a0 2024-04-25T13:04:12.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.358+0000 7fb7696d2640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb760009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.362+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb75c09c920 con 0x7fb7641029a0 2024-04-25T13:04:12.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:12 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2416658051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:12.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:12 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3840342578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:12.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:12 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1507615243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:12.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.547+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb764062b80 con 0x7fb7641029a0 2024-04-25T13:04:12.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.548+0000 7fb75a7fc640 1 -- 172.21.15.38:0/458804788 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fb75c09c220 con 0x7fb7641029a0 2024-04-25T13:04:12.547 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 msgr2=0x7fb7500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb760009520 comp rx=0 tx=0).stop 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 msgr2=0x7fb76419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 0x7fb76419f650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00d750 tx=0x7fb75c00dc10 comp rx=0 tx=0).stop 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 shutdown_connections 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7641029a0 0x7fb76419f650 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb764101fe0 0x7fb76419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 --2- 172.21.15.38:0/458804788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb764100e90 0x7fb764069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 >> 172.21.15.38:0/458804788 conn(0x7fb7640fc400 msgr2=0x7fb764105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 shutdown_connections 2024-04-25T13:04:12.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.550+0000 7fb76b936640 1 -- 172.21.15.38:0/458804788 wait complete. 2024-04-25T13:04:12.560 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:04:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:12 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2416658051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:12 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3840342578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:12 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1507615243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 -- 172.21.15.38:0/313289043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 msgr2=0x7f19d8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/313289043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8102460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009900 tx=0x7f19c002a410 comp rx=0 tx=0).stop 2024-04-25T13:04:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 -- 172.21.15.38:0/313289043 shutdown_connections 2024-04-25T13:04:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/313289043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d81029a0 0x7f19d81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/313289043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8102460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/313289043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 0x7f19d81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 -- 172.21.15.38:0/313289043 >> 172.21.15.38:0/313289043 conn(0x7f19d80fc400 msgr2=0x7f19d80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 -- 172.21.15.38:0/313289043 shutdown_connections 2024-04-25T13:04:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.663+0000 7f19dd1ed640 1 -- 172.21.15.38:0/313289043 wait complete. 2024-04-25T13:04:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 Processor -- start 2024-04-25T13:04:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 -- start start 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 0x7f19d81982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d81029a0 0x7f19d819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810dc40 con 0x7f19d8101fe0 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810dac0 con 0x7f19d81029a0 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19dd1ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810ddc0 con 0x7f19d8100e90 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40856/0 (socket says 172.21.15.38:40856) 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19d659c640 1 -- 172.21.15.38:0/3169129069 learned_addr learned my addr 172.21.15.38:0/3169129069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.664+0000 7f19d759e640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d81029a0 0x7f19d819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d6d9d640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 0x7f19d81982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d659c640 1 -- 172.21.15.38:0/3169129069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 msgr2=0x7f19d81982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d659c640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 0x7f19d81982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d659c640 1 -- 172.21.15.38:0/3169129069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d81029a0 msgr2=0x7f19d819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d659c640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d81029a0 0x7f19d819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d659c640 1 -- 172.21.15.38:0/3169129069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d819fef0 con 0x7f19d8101fe0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d6d9d640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 0x7f19d81982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19d659c640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8198810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f19c0006010 tx=0x7f19c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c003f030 con 0x7f19d8101fe0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d81a0180 con 0x7f19d8101fe0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19c0003cd0 con 0x7f19d8101fe0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0035070 con 0x7f19d8101fe0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.665+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d81a0670 con 0x7f19d8101fe0 2024-04-25T13:04:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.666+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d81081a0 con 0x7f19d8101fe0 2024-04-25T13:04:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.668+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f19c0042030 con 0x7f19d8101fe0 2024-04-25T13:04:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.668+0000 7f19cffff640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19c40a3bb0 0x7f19c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.669+0000 7f19d6d9d640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19c40a3bb0 0x7f19c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.669+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f19c00f0da0 con 0x7f19d8101fe0 2024-04-25T13:04:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.669+0000 7f19d6d9d640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19c40a3bb0 0x7f19c40a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19d00058c0 tx=0x7f19d0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:12.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.672+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19c00bd740 con 0x7f19d8101fe0 2024-04-25T13:04:12.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.899+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19d802d5c0 con 0x7f19d8101fe0 2024-04-25T13:04:12.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.900+0000 7f19cffff640 1 -- 172.21.15.38:0/3169129069 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11938 (secure 0 0 0) 0x7f19c00bd040 con 0x7f19d8101fe0 2024-04-25T13:04:12.899 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:12.899 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:02:51.699933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24484,24518,24482,24466,14625]},"id":1}]} 2024-04-25T13:04:12.899 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19c40a3bb0 msgr2=0x7f19c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19c40a3bb0 0x7f19c40a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19d00058c0 tx=0x7f19d0005d30 comp rx=0 tx=0).stop 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 msgr2=0x7f19d8198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8198810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f19c0006010 tx=0x7f19c00042f0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 shutdown_connections 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19c40a3bb0 0x7f19c40a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d81029a0 0x7f19d819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19d8101fe0 0x7f19d8198810 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 --2- 172.21.15.38:0/3169129069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d8100e90 0x7f19d81982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 >> 172.21.15.38:0/3169129069 conn(0x7f19d80fc400 msgr2=0x7f19d8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 shutdown_connections 2024-04-25T13:04:12.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:12.902+0000 7f19dd1ed640 1 -- 172.21.15.38:0/3169129069 wait complete. 2024-04-25T13:04:12.912 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:04:13.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2280107633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102a00 msgr2=0x7fcd74109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2280107633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102a00 0x7fcd74109320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009ca0 tx=0x7fcd6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2280107633 shutdown_connections 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2280107633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102a00 0x7fcd74109320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2280107633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74102040 0x7fcd741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2280107633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74100e60 0x7fcd74101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2280107633 >> 172.21.15.38:0/2280107633 conn(0x7fcd740fc440 msgr2=0x7fcd740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2280107633 shutdown_connections 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.016+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2280107633 wait complete. 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 Processor -- start 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 -- start start 2024-04-25T13:04:13.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74100e60 0x7fcd74198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 0x7fcd74198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74102a00 0x7fcd7419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7410cb40 con 0x7fcd74102040 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7410c9c0 con 0x7fcd74102a00 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7bf08640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7410ccc0 con 0x7fcd74100e60 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 0x7fcd74198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 0x7fcd74198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40868/0 (socket says 172.21.15.38:40868) 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 -- 172.21.15.38:0/2970146678 learned_addr learned my addr 172.21.15.38:0/2970146678 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd79ca4640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74100e60 0x7fcd74198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7a4a5640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74102a00 0x7fcd7419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 -- 172.21.15.38:0/2970146678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74100e60 msgr2=0x7fcd74198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74100e60 0x7fcd74198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 -- 172.21.15.38:0/2970146678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74102a00 msgr2=0x7fcd7419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74102a00 0x7fcd7419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd794a3640 1 -- 172.21.15.38:0/2970146678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7419fe40 con 0x7fcd74102040 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd79ca4640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74100e60 0x7fcd74198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.017+0000 7fcd7a4a5640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74102a00 0x7fcd7419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.018+0000 7fcd794a3640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 0x7fcd74198760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400d770 tx=0x7fcd6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.018+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64017070 con 0x7fcd74102040 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.018+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcd640054f0 con 0x7fcd74102040 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.018+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd741a0130 con 0x7fcd74102040 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.018+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64004de0 con 0x7fcd74102040 2024-04-25T13:04:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.018+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741a0680 con 0x7fcd74102040 2024-04-25T13:04:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.019+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd741a08e0 con 0x7fcd74102040 2024-04-25T13:04:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.020+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcd64004f80 con 0x7fcd74102040 2024-04-25T13:04:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.020+0000 7fcd6affd640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fcd600a3c30 0x7fcd600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.021+0000 7fcd79ca4640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fcd600a3c30 0x7fcd600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.021+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fcd640cfed0 con 0x7fcd74102040 2024-04-25T13:04:13.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.021+0000 7fcd79ca4640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fcd600a3c30 0x7fcd600a60f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005d00 tx=0x7fcd70005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.025+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd6409c8b0 con 0x7fcd74102040 2024-04-25T13:04:13.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.217+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd741a1110 con 0x7fcd74102040 2024-04-25T13:04:13.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.218+0000 7fcd6affd640 1 -- 172.21.15.38:0/2970146678 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22650 (secure 0 0 0) 0x7fcd6409c1b0 con 0x7fcd74102040 2024-04-25T13:04:13.217 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:13.217 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:03:37.115660+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:04:13.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.220+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fcd600a3c30 msgr2=0x7fcd600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.220+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fcd600a3c30 0x7fcd600a60f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005d00 tx=0x7fcd70005c90 comp rx=0 tx=0).stop 2024-04-25T13:04:13.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.220+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 msgr2=0x7fcd74198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.220+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 0x7fcd74198760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400d770 tx=0x7fcd6400dc30 comp rx=0 tx=0).stop 2024-04-25T13:04:13.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.220+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 shutdown_connections 2024-04-25T13:04:13.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.220+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fcd600a3c30 0x7fcd600a60f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.221+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd74102a00 0x7fcd7419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.221+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd74102040 0x7fcd74198760 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.221+0000 7fcd7bf08640 1 --2- 172.21.15.38:0/2970146678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd74100e60 0x7fcd74198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.221+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 >> 172.21.15.38:0/2970146678 conn(0x7fcd740fc440 msgr2=0x7fcd7406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.221+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 shutdown_connections 2024-04-25T13:04:13.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.221+0000 7fcd7bf08640 1 -- 172.21.15.38:0/2970146678 wait complete. 2024-04-25T13:04:13.229 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:13 smithi038 ceph-mon[29400]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:13.229 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:13 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1932410101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:13.229 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:13 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/458804788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:13.229 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:13 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3169129069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:13.231 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.335+0000 7f18a4319640 1 -- 172.21.15.38:0/996401592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103b50 msgr2=0x7f189c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.335+0000 7f18a4319640 1 --2- 172.21.15.38:0/996401592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103b50 0x7f189c10a3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1894009ca0 tx=0x7f189402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- 172.21.15.38:0/996401592 shutdown_connections 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 --2- 172.21.15.38:0/996401592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103b50 0x7f189c10a3e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 --2- 172.21.15.38:0/996401592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189c103190 0x7f189c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 --2- 172.21.15.38:0/996401592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c101fa0 0x7f189c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- 172.21.15.38:0/996401592 >> 172.21.15.38:0/996401592 conn(0x7f189c0fd530 msgr2=0x7f189c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- 172.21.15.38:0/996401592 shutdown_connections 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- 172.21.15.38:0/996401592 wait complete. 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 Processor -- start 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- start start 2024-04-25T13:04:13.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c101fa0 0x7f189c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 0x7f189c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189c103b50 0x7f189c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10dc00 con 0x7f189c103190 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10da80 con 0x7f189c101fa0 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.336+0000 7f18a4319640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10dd80 con 0x7f189c103b50 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 0x7f189c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 0x7f189c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40894/0 (socket says 172.21.15.38:40894) 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 -- 172.21.15.38:0/961199512 learned_addr learned my addr 172.21.15.38:0/961199512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a20b5640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c101fa0 0x7f189c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 -- 172.21.15.38:0/961199512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189c103b50 msgr2=0x7f189c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189c103b50 0x7f189c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 -- 172.21.15.38:0/961199512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c101fa0 msgr2=0x7f189c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c101fa0 0x7f189c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 -- 172.21.15.38:0/961199512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c19fde0 con 0x7f189c103190 2024-04-25T13:04:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a18b4640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 0x7f189c198700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c920 tx=0x7f188c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c007c70 con 0x7f189c103190 2024-04-25T13:04:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f188c007e10 con 0x7f189c103190 2024-04-25T13:04:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a00d0 con 0x7f189c103190 2024-04-25T13:04:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0188c0 con 0x7f189c103190 2024-04-25T13:04:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.337+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a0620 con 0x7f189c103190 2024-04-25T13:04:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.338+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c1092c0 con 0x7f189c103190 2024-04-25T13:04:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.339+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f188c018a60 con 0x7f189c103190 2024-04-25T13:04:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.340+0000 7f18937fe640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f18880a3bb0 0x7f18880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.340+0000 7f18a20b5640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f18880a3bb0 0x7f18880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.340+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f188c0d05f0 con 0x7f189c103190 2024-04-25T13:04:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.340+0000 7f18a20b5640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f18880a3bb0 0x7f18880a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f189c102ff0 tx=0x7f1898005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.344+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f188c014030 con 0x7f189c103190 2024-04-25T13:04:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:13 smithi178 ceph-mon[33233]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:13 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1932410101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:13 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/458804788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:13.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:13 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3169129069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.569+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f189c062bf0 con 0x7f189c103190 2024-04-25T13:04:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.570+0000 7f18937fe640 1 -- 172.21.15.38:0/961199512 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11938 (secure 0 0 0) 0x7f188c00de20 con 0x7f189c103190 2024-04-25T13:04:13.569 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:13.569 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:02:51.699933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24484,24518,24482,24466,14625]},"id":1}]} 2024-04-25T13:04:13.569 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f18880a3bb0 msgr2=0x7f18880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f18880a3bb0 0x7f18880a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f189c102ff0 tx=0x7f1898005f90 comp rx=0 tx=0).stop 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 msgr2=0x7f189c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 0x7f189c198700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c920 tx=0x7f188c00cde0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 shutdown_connections 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f18880a3bb0 0x7f18880a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189c103b50 0x7f189c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f189c103190 0x7f189c198700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 --2- 172.21.15.38:0/961199512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c101fa0 0x7f189c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.572+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 >> 172.21.15.38:0/961199512 conn(0x7f189c0fd530 msgr2=0x7f189c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.573+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 shutdown_connections 2024-04-25T13:04:13.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.573+0000 7f18a4319640 1 -- 172.21.15.38:0/961199512 wait complete. 2024-04-25T13:04:13.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:13 smithi104 ceph-mon[32726]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:13.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:13 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1932410101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:13.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:13 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/458804788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:13.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:13 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3169129069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:13.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:02:44.261324+0000', 'modified': '2024-04-25T13:02:51.699933+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24518, 'mds_1': 24484, 'mds_2': 24482, 'mds_3': 24466, 'mds_4': 14625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/3427364907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 3427364907}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 3427364907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.38:6841/4187315479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 4187315479}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 4187315479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/2494647672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 2494647672}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 2494647672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/650369302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 650369302}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 650369302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/4172911078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 4172911078}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 4172911078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/2362992725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 2362992725}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2362992725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.178:6837/2053792208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2053792208}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2053792208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.104:6837/2247876378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2247876378}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2247876378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.178:6839/391098246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 391098246}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 391098246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/1961595450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1961595450}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1961595450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24484, 24518, 24482, 24466, 14625]} 2024-04-25T13:04:13.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T13:04:13.583 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 -- 172.21.15.38:0/1882654117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e41031b0 msgr2=0x7f19e4103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/1882654117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e41031b0 0x7f19e4103630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 -- 172.21.15.38:0/1882654117 shutdown_connections 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/1882654117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4103b70 0x7f19e410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/1882654117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e41031b0 0x7f19e4103630 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/1882654117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e4101fc0 0x7f19e41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 -- 172.21.15.38:0/1882654117 >> 172.21.15.38:0/1882654117 conn(0x7f19e40fd530 msgr2=0x7f19e40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 -- 172.21.15.38:0/1882654117 shutdown_connections 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.687+0000 7f19ec2e1640 1 -- 172.21.15.38:0/1882654117 wait complete. 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 Processor -- start 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 -- start start 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4101fc0 0x7f19e41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e41031b0 0x7f19e4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 0x7f19e419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410dc20 con 0x7f19e4103b70 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410daa0 con 0x7f19e4101fc0 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ec2e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410dda0 con 0x7f19e41031b0 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 0x7f19e419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 0x7f19e419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40910/0 (socket says 172.21.15.38:40910) 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 -- 172.21.15.38:0/2365533899 learned_addr learned my addr 172.21.15.38:0/2365533899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19e987c640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e41031b0 0x7f19e4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4101fc0 0x7f19e41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 -- 172.21.15.38:0/2365533899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e41031b0 msgr2=0x7f19e4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e41031b0 0x7f19e4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 -- 172.21.15.38:0/2365533899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4101fc0 msgr2=0x7f19e41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4101fc0 0x7f19e41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea87e640 1 -- 172.21.15.38:0/2365533899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e419fee0 con 0x7f19e4103b70 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4101fc0 0x7f19e41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.688+0000 7f19e987c640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e41031b0 0x7f19e4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.689+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 0x7f19e419f870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d950 tx=0x7f19d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.689+0000 7f19e37fe640 1 -- 172.21.15.38:0/2365533899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4017030 con 0x7f19e4103b70 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.689+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41a01d0 con 0x7f19e4103b70 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.689+0000 7f19e37fe640 1 -- 172.21.15.38:0/2365533899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19d4004020 con 0x7f19e4103b70 2024-04-25T13:04:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.689+0000 7f19e37fe640 1 -- 172.21.15.38:0/2365533899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4003bc0 con 0x7f19e4103b70 2024-04-25T13:04:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.689+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a06d0 con 0x7f19e4103b70 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.691+0000 7f19e37fe640 1 -- 172.21.15.38:0/2365533899 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f19d401a070 con 0x7f19e4103b70 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.691+0000 7f19eeff3740 1 Processor -- start 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 -- start start 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19e37fe640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19cc0a3bb0 0x7f19cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f71113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f71193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7071c50 con 0x55e4f7110f90 2024-04-25T13:04:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4f700e830 con 0x55e4f7112b20 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19eeff3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e4f706dc10 con 0x55e4f7112160 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19e987c640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19cc0a3bb0 0x7f19cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19e37fe640 1 -- 172.21.15.38:0/2365533899 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f19d40cfca0 con 0x7f19e4103b70 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f71113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f71113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40920/0 (socket says 172.21.15.38:40920) 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 -- 172.21.15.38:0/2807078531 learned_addr learned my addr 172.21.15.38:0/2807078531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19e987c640 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f71193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 -- 172.21.15.38:0/2807078531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 msgr2=0x55e4f71125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 -- 172.21.15.38:0/2807078531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 msgr2=0x55e4f71193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f71193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.692+0000 7f19ea07d640 1 -- 172.21.15.38:0/2807078531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f6d842d0 con 0x55e4f7110f90 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19e987c640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19cc0a3bb0 0x7f19cc0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19e41997d0 tx=0x7f19d80316f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f71113b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f19dc010400 tx=0x7f19dc0108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea1d470ea7dff35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19e0bfe640 1 -- 172.21.15.38:0/2807078531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc003b70 con 0x55e4f7110f90 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19e0bfe640 1 -- 172.21.15.38:0/2807078531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19dc003d10 con 0x55e4f7110f90 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19e0bfe640 1 -- 172.21.15.38:0/2807078531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011ac0 con 0x55e4f7110f90 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19eeff3740 1 -- 172.21.15.38:0/2807078531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 msgr2=0x55e4f71113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.693+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f71113b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f19dc010400 tx=0x7f19dc0108c0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- 172.21.15.38:0/2807078531 shutdown_connections 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f71193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2807078531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f71113b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- 172.21.15.38:0/2807078531 >> 172.21.15.38:0/2807078531 conn(0x55e4f7108070 msgr2=0x55e4f7108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- 172.21.15.38:0/2807078531 shutdown_connections 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- 172.21.15.38:0/2807078531 wait complete. 2024-04-25T13:04:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- start start 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 Processor -- start 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f711b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112160 0x55e4f711b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112b20 0x55e4f7122930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7119b30 con 0x55e4f7110f90 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4f700c470 con 0x55e4f7112160 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7119cb0 con 0x55e4f7112b20 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19ea07d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f711b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19eeff3740 10 client.0 _ll_register_callbacks cb 0x55e4f7031f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19ea07d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f711b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40926/0 (socket says 172.21.15.38:40926) 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19ea07d640 1 -- 172.21.15.38:0/3793902171 learned_addr learned my addr 172.21.15.38:0/3793902171 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.694+0000 7f19e987c640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112160 0x55e4f711b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea87e640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112b20 0x55e4f7122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea07d640 1 -- 172.21.15.38:0/3793902171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112b20 msgr2=0x55e4f7122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea07d640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112b20 0x55e4f7122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea07d640 1 -- 172.21.15.38:0/3793902171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112160 msgr2=0x55e4f711b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea07d640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112160 0x55e4f711b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea07d640 1 -- 172.21.15.38:0/3793902171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f6e4e6b0 con 0x55e4f7110f90 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea87e640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112b20 0x55e4f7122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19ea07d640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f711b3b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0103d0 tx=0x7f19dc004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db9ce28ec365c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19c9ffb640 1 -- 172.21.15.38:0/3793902171 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011c50 con 0x55e4f7110f90 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19c9ffb640 1 -- 172.21.15.38:0/3793902171 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f19dc011df0 con 0x55e4f7110f90 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19dc003d10 con 0x55e4f7110f90 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19c9ffb640 1 -- 172.21.15.38:0/3793902171 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19dc003ec0 con 0x55e4f7110f90 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19eeff3740 10 client.15045 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4f7125ca0 con 0x55e4f7110f90 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.695+0000 7f19c9ffb640 1 -- 172.21.15.38:0/3793902171 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19dc003d10 con 0x55e4f7110f90 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19c9ffb640 1 -- 172.21.15.38:0/3793902171 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19dc005450 con 0x55e4f7110f90 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19eeff3740 10 client.15045 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19eeff3740 10 client.15045 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19eeff3740 20 client.15045 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19eeff3740 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7127f20 0x55e4f7129390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19eeff3740 4 client.15045 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 --> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4f707f670 con 0x55e4f7127f20 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19e987c640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7127f20 0x55e4f7129390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19c9ffb640 10 client.15045 ms_handle_connect on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.696+0000 7f19e987c640 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7127f20 0x55e4f7129390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71fedf2d5fe288d4 server_cookie=4110d4feebdd1667 in_seq=0 out_seq=0 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19c9ffb640 1 -- 172.21.15.38:0/3793902171 <== mds.4 v2:172.21.15.38:6840/4187315479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4f707f670 con 0x55e4f7127f20 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19c9ffb640 10 client.15045 handle_command_reply: tid=0 2024-04-25T13:04:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 client.15045 shutdown 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 msgr2=0x55e4f711b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f711b3b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0103d0 tx=0x7f19dc004cf0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 shutdown_connections 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7127f20 0x55e4f7129390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112b20 0x55e4f7122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112160 0x55e4f711b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19c9ffb640 0 client.15045 ms_handle_reset on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 --2- 172.21.15.38:0/3793902171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7110f90 0x55e4f711b3b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 >> 172.21.15.38:0/3793902171 conn(0x55e4f7108070 msgr2=0x55e4f71071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 shutdown_connections 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 1 -- 172.21.15.38:0/3793902171 wait complete. 2024-04-25T13:04:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.697+0000 7f19eeff3740 20 client.15045 trim_cache size 0 max 16384 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 Processor -- start 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 -- start start 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7110f90 0x55e4f71030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 0x55e4f71071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e4f71276d0 con 0x55e4f7112b20 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7127da0 con 0x55e4f7110f90 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19eeff3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7128470 con 0x55e4f7112160 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19ea87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 0x55e4f71071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19ea87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 0x55e4f71071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40934/0 (socket says 172.21.15.38:40934) 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.715+0000 7f19ea87e640 1 -- 172.21.15.38:0/2207774424 learned_addr learned my addr 172.21.15.38:0/2207774424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7110f90 0x55e4f71030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea87e640 1 -- 172.21.15.38:0/2207774424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 msgr2=0x55e4f71228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19e987c640 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71228f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea87e640 1 -- 172.21.15.38:0/2207774424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7110f90 msgr2=0x55e4f71030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7110f90 0x55e4f71030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea87e640 1 -- 172.21.15.38:0/2207774424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f7111be0 con 0x55e4f7112b20 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea07d640 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7110f90 0x55e4f71030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19ea87e640 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 0x55e4f71071d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f19d4098950 tx=0x7f19d400c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf7241656ea5e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19cbfff640 1 -- 172.21.15.38:0/2207774424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d401e110 con 0x55e4f7112b20 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19cbfff640 1 -- 172.21.15.38:0/2207774424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19d409bf80 con 0x55e4f7112b20 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.716+0000 7f19cbfff640 1 -- 172.21.15.38:0/2207774424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d40041d0 con 0x55e4f7112b20 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- 172.21.15.38:0/2207774424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 msgr2=0x55e4f71071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 0x55e4f71071d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f19d4098950 tx=0x7f19d400c5b0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- 172.21.15.38:0/2207774424 shutdown_connections 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112b20 0x55e4f71071d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7112160 0x55e4f71228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- 172.21.15.38:0/2207774424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7110f90 0x55e4f71030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- 172.21.15.38:0/2207774424 >> 172.21.15.38:0/2207774424 conn(0x55e4f7108070 msgr2=0x55e4f7108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- 172.21.15.38:0/2207774424 shutdown_connections 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- 172.21.15.38:0/2207774424 wait complete. 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- start start 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 Processor -- start 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7110f90 0x55e4f7120840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 0x55e4f7189eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f718c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7129210 con 0x55e4f7112160 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7128b40 con 0x55e4f7112b20 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e4f71298e0 con 0x55e4f7110f90 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19eeff3740 10 client.0 _ll_register_callbacks cb 0x7f19d400b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19e987c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 0x55e4f7189eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19e987c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 0x55e4f7189eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40950/0 (socket says 172.21.15.38:40950) 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.717+0000 7f19e987c640 1 -- 172.21.15.38:0/1532568753 learned_addr learned my addr 172.21.15.38:0/1532568753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19ea07d640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7110f90 0x55e4f7120840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19ea87e640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f718c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19e987c640 1 -- 172.21.15.38:0/1532568753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7110f90 msgr2=0x55e4f7120840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19e987c640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7110f90 0x55e4f7120840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19e987c640 1 -- 172.21.15.38:0/1532568753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 msgr2=0x55e4f718c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19e987c640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f718c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19e987c640 1 -- 172.21.15.38:0/1532568753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f718ccb0 con 0x55e4f7112160 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19e987c640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 0x55e4f7189eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19d802a930 tx=0x7f19d802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d43191beb3f47d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19c9ffb640 1 -- 172.21.15.38:0/1532568753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8057060 con 0x55e4f7112160 2024-04-25T13:04:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19c9ffb640 1 -- 172.21.15.38:0/1532568753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f19d8004380 con 0x55e4f7112160 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d409bf80 con 0x55e4f7112160 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19c9ffb640 1 -- 172.21.15.38:0/1532568753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19d8035050 con 0x55e4f7112160 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19eeff3740 10 client.15051 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4f718e080 con 0x55e4f7112160 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.718+0000 7f19c9ffb640 1 -- 172.21.15.38:0/1532568753 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19d409bf80 con 0x55e4f7112160 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19c9ffb640 1 -- 172.21.15.38:0/1532568753 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19d805c070 con 0x55e4f7112160 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 10 client.15051 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 10 client.15051 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 20 client.15051 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7192c70 0x55e4f7191980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 4 client.15051 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 --> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] -- command(tid 0: {"prefix": "status"}) -- 0x55e4f70794e0 con 0x55e4f7192c70 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19ea07d640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7192c70 0x55e4f7191980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19c9ffb640 10 client.15051 ms_handle_connect on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19ea07d640 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7192c70 0x55e4f7191980 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e5467da958b18b2a server_cookie=dae66113aa48bccf in_seq=0 out_seq=0 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19c9ffb640 1 -- 172.21.15.38:0/1532568753 <== mds.4 v2:172.21.15.38:6840/4187315479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e4f70794e0 con 0x55e4f7192c70 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19c9ffb640 10 client.15051 handle_command_reply: tid=0 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 client.15051 shutdown 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 msgr2=0x55e4f7189eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 0x55e4f7189eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19d802a930 tx=0x7f19d802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 shutdown_connections 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x55e4f7192c70 0x55e4f7191980 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4f7112b20 0x55e4f718c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4f7112160 0x55e4f7189eb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19c9ffb640 0 client.15051 ms_handle_reset on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 --2- 172.21.15.38:0/1532568753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4f7110f90 0x55e4f7120840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 >> 172.21.15.38:0/1532568753 conn(0x55e4f7108070 msgr2=0x55e4f71030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 shutdown_connections 2024-04-25T13:04:13.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.719+0000 7f19eeff3740 1 -- 172.21.15.38:0/1532568753 wait complete. 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19eeff3740 20 client.15051 trim_cache size 0 max 16384 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 14625, 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 83.028130281000003, 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:13.725 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:13.726 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 95.987471197999994 2024-04-25T13:04:13.726 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:13.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19cc0a3bb0 msgr2=0x7f19cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19cc0a3bb0 0x7f19cc0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19e41997d0 tx=0x7f19d80316f0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 msgr2=0x7f19e419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 0x7f19e419f870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d950 tx=0x7f19d400de10 comp rx=0 tx=0).stop 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 shutdown_connections 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f19cc0a3bb0 0x7f19cc0a6070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19e4103b70 0x7f19e419f870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e41031b0 0x7f19e4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 --2- 172.21.15.38:0/2365533899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e4101fc0 0x7f19e41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 >> 172.21.15.38:0/2365533899 conn(0x7f19e40fd530 msgr2=0x7f19e4106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 shutdown_connections 2024-04-25T13:04:13.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.720+0000 7f19ec2e1640 1 -- 172.21.15.38:0/2365533899 wait complete. 2024-04-25T13:04:13.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 -- 172.21.15.38:0/1656868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c1031b0 msgr2=0x7f343c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 --2- 172.21.15.38:0/1656868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c1031b0 0x7f343c103630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f342c009900 tx=0x7f342c02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 -- 172.21.15.38:0/1656868943 shutdown_connections 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 --2- 172.21.15.38:0/1656868943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c103b70 0x7f343c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 --2- 172.21.15.38:0/1656868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c1031b0 0x7f343c103630 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 --2- 172.21.15.38:0/1656868943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f343c101fc0 0x7f343c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 -- 172.21.15.38:0/1656868943 >> 172.21.15.38:0/1656868943 conn(0x7f343c0fd530 msgr2=0x7f343c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 -- 172.21.15.38:0/1656868943 shutdown_connections 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.834+0000 7f3444205640 1 -- 172.21.15.38:0/1656868943 wait complete. 2024-04-25T13:04:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 Processor -- start 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 -- start start 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 0x7f343c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c1031b0 0x7f343c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f343c103b70 0x7f343c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f343c10dc20 con 0x7f343c101fc0 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f343c10daa0 con 0x7f343c103b70 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3444205640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f343c10dda0 con 0x7f343c1031b0 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 0x7f343c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 0x7f343c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40966/0 (socket says 172.21.15.38:40966) 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 -- 172.21.15.38:0/2622554036 learned_addr learned my addr 172.21.15.38:0/2622554036 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f34427a2640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f343c103b70 0x7f343c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f34417a0640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c1031b0 0x7f343c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 -- 172.21.15.38:0/2622554036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c1031b0 msgr2=0x7f343c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c1031b0 0x7f343c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 -- 172.21.15.38:0/2622554036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f343c103b70 msgr2=0x7f343c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f343c103b70 0x7f343c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.835+0000 7f3441fa1640 1 -- 172.21.15.38:0/2622554036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c19fe80 con 0x7f343c101fc0 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.836+0000 7f3441fa1640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 0x7f343c198260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f343800d770 tx=0x7f343800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.836+0000 7f3432ffd640 1 -- 172.21.15.38:0/2622554036 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438017070 con 0x7f343c101fc0 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.836+0000 7f3432ffd640 1 -- 172.21.15.38:0/2622554036 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3438009c30 con 0x7f343c101fc0 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.836+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c1a0170 con 0x7f343c101fc0 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.836+0000 7f3432ffd640 1 -- 172.21.15.38:0/2622554036 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438003d70 con 0x7f343c101fc0 2024-04-25T13:04:13.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.836+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c1a06c0 con 0x7f343c101fc0 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3432ffd640 1 -- 172.21.15.38:0/2622554036 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3438003f10 con 0x7f343c101fc0 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 Processor -- start 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 -- start start 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 0x55feb91e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55feb913dc50 con 0x55feb91da710 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55feb90da830 con 0x55feb91d9d50 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3446ff9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55feb9139c10 con 0x55feb91d8b80 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f34417a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 0x55feb91e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f34417a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 0x55feb91e0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40968/0 (socket says 172.21.15.38:40968) 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f34417a0640 1 -- 172.21.15.38:0/3019863420 learned_addr learned my addr 172.21.15.38:0/3019863420 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f34427a2640 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3441fa1640 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.839+0000 7f3432ffd640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f34280a3bb0 0x7f34280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34417a0640 1 -- 172.21.15.38:0/3019863420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 msgr2=0x55feb91d8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34417a0640 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91d8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34417a0640 1 -- 172.21.15.38:0/3019863420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 msgr2=0x55feb91da1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34427a2640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f34280a3bb0 0x7f34280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34417a0640 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91da1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34417a0640 1 -- 172.21.15.38:0/3019863420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb8e502d0 con 0x55feb91da710 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f3441fa1640 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91da1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f3432ffd640 1 -- 172.21.15.38:0/2622554036 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f34380cfd70 con 0x7f343c101fc0 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34417a0640 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 0x55feb91e0fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f342c02a930 tx=0x7f342c032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7064be880aa066d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f3427fff640 1 -- 172.21.15.38:0/3019863420 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c03f070 con 0x55feb91da710 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f34427a2640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f34280a3bb0 0x7f34280a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55feb91d9c50 tx=0x7f3434004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f3427fff640 1 -- 172.21.15.38:0/3019863420 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f342c004030 con 0x55feb91da710 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.840+0000 7f3427fff640 1 -- 172.21.15.38:0/3019863420 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c035070 con 0x55feb91da710 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 -- 172.21.15.38:0/3019863420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 msgr2=0x55feb91e0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 0x55feb91e0fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f342c02a930 tx=0x7f342c032b50 comp rx=0 tx=0).stop 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 -- 172.21.15.38:0/3019863420 shutdown_connections 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91da710 0x55feb91e0fa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91da1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 --2- 172.21.15.38:0/3019863420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 -- 172.21.15.38:0/3019863420 >> 172.21.15.38:0/3019863420 conn(0x55feb91d4070 msgr2=0x55feb91d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 -- 172.21.15.38:0/3019863420 shutdown_connections 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 -- 172.21.15.38:0/3019863420 wait complete. 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 -- start start 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.841+0000 7f3446ff9740 1 Processor -- start 2024-04-25T13:04:13.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91da710 0x55feb91ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55feb91e16c0 con 0x55feb91d9d50 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55feb90d8470 con 0x55feb91da710 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55feb91e1840 con 0x55feb91d8b80 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 10 client.0 _ll_register_callbacks cb 0x55feb90fdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91e34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40980/0 (socket says 172.21.15.38:40980) 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 -- 172.21.15.38:0/782564077 learned_addr learned my addr 172.21.15.38:0/782564077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3441fa1640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34427a2640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91da710 0x55feb91ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 -- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 msgr2=0x55feb91e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 -- 172.21.15.38:0/782564077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91da710 msgr2=0x55feb91ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91da710 0x55feb91ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 -- 172.21.15.38:0/782564077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb8f1a6b0 con 0x55feb91d9d50 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34427a2640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91da710 0x55feb91ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34417a0640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91e34c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f342c004490 tx=0x7f342c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed1dbb42c17f57a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34257fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34257fa640 1 -- 172.21.15.38:0/782564077 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c04b020 con 0x55feb91d9d50 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f34257fa640 1 -- 172.21.15.38:0/782564077 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f342c0035f0 con 0x55feb91d9d50 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.842+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c004030 con 0x55feb91d9d50 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f34257fa640 1 -- 172.21.15.38:0/782564077 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f342c035070 con 0x55feb91d9d50 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 10 client.15063 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55feb91ed870 con 0x55feb91d9d50 2024-04-25T13:04:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f34257fa640 1 -- 172.21.15.38:0/782564077 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f342c004030 con 0x55feb91d9d50 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f34257fa640 1 -- 172.21.15.38:0/782564077 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f342c03bb40 con 0x55feb91d9d50 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 20 client.15063 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb91efa10 0x55feb91f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 4 client.15063 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 --> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb914b670 con 0x55feb91efa10 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.843+0000 7f3441fa1640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb91efa10 0x55feb91f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.844+0000 7f34257fa640 10 client.15063 ms_handle_connect on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.844+0000 7f3441fa1640 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb91efa10 0x55feb91f0e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=527418a5be2d6d29 server_cookie=c16c7a3a6bea59a8 in_seq=0 out_seq=0 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.844+0000 7f34257fa640 1 -- 172.21.15.38:0/782564077 <== mds.3 v2:172.21.15.178:6836/2053792208 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55feb914b670 con 0x55feb91efa10 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f34257fa640 10 client.15063 handle_command_reply: tid=0 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 client.15063 shutdown 2024-04-25T13:04:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 msgr2=0x55feb91e34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91e34c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f342c004490 tx=0x7f342c0044c0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 shutdown_connections 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb91efa10 0x55feb91f0e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91da710 0x55feb91ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91e34c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f34257fa640 0 client.15063 ms_handle_reset on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 --2- 172.21.15.38:0/782564077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91d8b80 0x55feb91e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 >> 172.21.15.38:0/782564077 conn(0x55feb91d4070 msgr2=0x55feb91d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 shutdown_connections 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 1 -- 172.21.15.38:0/782564077 wait complete. 2024-04-25T13:04:13.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.845+0000 7f3446ff9740 20 client.15063 trim_cache size 0 max 16384 2024-04-25T13:04:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 Processor -- start 2024-04-25T13:04:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 -- start start 2024-04-25T13:04:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d8b80 0x55feb91d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55feb91ef1c0 con 0x55feb91d9d50 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55feb91ef890 con 0x55feb91d8b80 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.863+0000 7f3446ff9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55feb91eff60 con 0x55feb91da710 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40990/0 (socket says 172.21.15.38:40990) 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 -- 172.21.15.38:0/924460548 learned_addr learned my addr 172.21.15.38:0/924460548 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34427a2640 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 -- 172.21.15.38:0/924460548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 msgr2=0x55feb91d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 -- 172.21.15.38:0/924460548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d8b80 msgr2=0x55feb91d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d8b80 0x55feb91d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 -- 172.21.15.38:0/924460548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb91cf960 con 0x55feb91d9d50 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34427a2640 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34417a0640 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91ea4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f342c02a930 tx=0x7f342c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a892c2c9d41c24d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34277fe640 1 -- 172.21.15.38:0/924460548 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c04b040 con 0x55feb91d9d50 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34277fe640 1 -- 172.21.15.38:0/924460548 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f342c052e50 con 0x55feb91d9d50 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.864+0000 7f34277fe640 1 -- 172.21.15.38:0/924460548 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c03f040 con 0x55feb91d9d50 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 -- 172.21.15.38:0/924460548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 msgr2=0x55feb91ea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91ea4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f342c02a930 tx=0x7f342c005a20 comp rx=0 tx=0).stop 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 -- 172.21.15.38:0/924460548 shutdown_connections 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d9d50 0x55feb91ea4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 --2- 172.21.15.38:0/924460548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d8b80 0x55feb91d6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 -- 172.21.15.38:0/924460548 >> 172.21.15.38:0/924460548 conn(0x55feb91d4070 msgr2=0x55feb91d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 -- 172.21.15.38:0/924460548 shutdown_connections 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 -- 172.21.15.38:0/924460548 wait complete. 2024-04-25T13:04:13.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 -- start start 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.865+0000 7f3446ff9740 1 Processor -- start 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 0x55feb91e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55feb91f0d00 con 0x55feb91d8b80 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55feb91f0630 con 0x55feb91d9d50 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55feb91f13d0 con 0x55feb91da710 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 10 client.0 _ll_register_callbacks cb 0x7f3434009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 0x55feb91e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 0x55feb91e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40994/0 (socket says 172.21.15.38:40994) 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 -- 172.21.15.38:0/1539587592 learned_addr learned my addr 172.21.15.38:0/1539587592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f34417a0640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 -- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 msgr2=0x55feb91e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 -- 172.21.15.38:0/1539587592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 msgr2=0x55feb91e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 -- 172.21.15.38:0/1539587592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb91e9870 con 0x55feb91d8b80 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f34417a0640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3441fa1640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 0x55feb91e5d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f34380984b0 tx=0x7f343800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db0230016d57ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f34257fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f34257fa640 1 -- 172.21.15.38:0/1539587592 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438020040 con 0x55feb91d8b80 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f34257fa640 1 -- 172.21.15.38:0/1539587592 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f34380cd980 con 0x55feb91d8b80 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c052e50 con 0x55feb91d8b80 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f34257fa640 1 -- 172.21.15.38:0/1539587592 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34380a0400 con 0x55feb91d8b80 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 10 client.15069 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.866+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55feb9255b80 con 0x55feb91d8b80 2024-04-25T13:04:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f34257fa640 1 -- 172.21.15.38:0/1539587592 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f342c052e50 con 0x55feb91d8b80 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f34257fa640 1 -- 172.21.15.38:0/1539587592 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f342c0054d0 con 0x55feb91d8b80 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f3446ff9740 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f3446ff9740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f3446ff9740 20 client.15069 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f3446ff9740 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb925a790 0x55feb92594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f3446ff9740 4 client.15069 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 --> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] -- command(tid 0: {"prefix": "status"}) -- 0x55feb91454e0 con 0x55feb925a790 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f34417a0640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb925a790 0x55feb92594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f34257fa640 10 client.15069 ms_handle_connect on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.867+0000 7f34417a0640 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb925a790 0x55feb92594a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcecc42ec4d79191 server_cookie=ac45dd5a8c7a098f in_seq=0 out_seq=0 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f34257fa640 1 -- 172.21.15.38:0/1539587592 <== mds.3 v2:172.21.15.178:6836/2053792208 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55feb91454e0 con 0x55feb925a790 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f34257fa640 10 client.15069 handle_command_reply: tid=0 2024-04-25T13:04:13.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 client.15069 shutdown 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 msgr2=0x55feb91e5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 0x55feb91e5d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f34380984b0 tx=0x7f343800df30 comp rx=0 tx=0).stop 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 shutdown_connections 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55feb925a790 0x55feb92594a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55feb91da710 0x55feb91e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f34257fa640 0 client.15069 ms_handle_reset on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55feb91d9d50 0x55feb91e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 --2- 172.21.15.38:0/1539587592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb91d8b80 0x55feb91e5d90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 >> 172.21.15.38:0/1539587592 conn(0x55feb91d4070 msgr2=0x55feb91d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 shutdown_connections 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 1 -- 172.21.15.38:0/1539587592 wait complete. 2024-04-25T13:04:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.868+0000 7f3446ff9740 20 client.15069 trim_cache size 0 max 16384 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24466, 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 83.908562144000001, 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:13.874 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 107.555599376 2024-04-25T13:04:13.875 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:13.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f34280a3bb0 msgr2=0x7f34280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f34280a3bb0 0x7f34280a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55feb91d9c50 tx=0x7f3434004ab0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 msgr2=0x7f343c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 0x7f343c198260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f343800d770 tx=0x7f343800dc30 comp rx=0 tx=0).stop 2024-04-25T13:04:13.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 shutdown_connections 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f34280a3bb0 0x7f34280a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f343c103b70 0x7f343c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c1031b0 0x7f343c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 --2- 172.21.15.38:0/2622554036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c101fc0 0x7f343c198260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 >> 172.21.15.38:0/2622554036 conn(0x7f343c0fd530 msgr2=0x7f343c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 shutdown_connections 2024-04-25T13:04:13.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.869+0000 7f3444205640 1 -- 172.21.15.38:0/2622554036 wait complete. 2024-04-25T13:04:13.878 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T13:04:13.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 -- 172.21.15.38:0/3747487132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c103b70 msgr2=0x7f4d1c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3747487132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c103b70 0x7f4d1c10a400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009ca0 tx=0x7f4d1402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 -- 172.21.15.38:0/3747487132 shutdown_connections 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3747487132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c103b70 0x7f4d1c10a400 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3747487132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3747487132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 -- 172.21.15.38:0/3747487132 >> 172.21.15.38:0/3747487132 conn(0x7f4d1c0fd530 msgr2=0x7f4d1c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 -- 172.21.15.38:0/3747487132 shutdown_connections 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 -- 172.21.15.38:0/3747487132 wait complete. 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 Processor -- start 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.982+0000 7f4d2259b640 1 -- start start 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d2259b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d2259b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d2259b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d2259b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10dc00 con 0x7f4d1c101fc0 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d2259b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10da80 con 0x7f4d1c1031b0 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d2259b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10dd80 con 0x7f4d1c103b70 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41014/0 (socket says 172.21.15.38:41014) 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3237039028 learned_addr learned my addr 172.21.15.38:0/3237039028 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1b7fe640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d20b38640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3237039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c103b70 msgr2=0x7f4d1c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3237039028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d1c1031b0 msgr2=0x7f4d1c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3237039028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c19fe50 con 0x7f4d1c101fc0 2024-04-25T13:04:13.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c970 tx=0x7f4d0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.983+0000 7f4d197fa640 1 -- 172.21.15.38:0/3237039028 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c007c10 con 0x7f4d1c101fc0 2024-04-25T13:04:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.984+0000 7f4d197fa640 1 -- 172.21.15.38:0/3237039028 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4d0c007db0 con 0x7f4d1c101fc0 2024-04-25T13:04:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.984+0000 7f4d197fa640 1 -- 172.21.15.38:0/3237039028 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00f910 con 0x7f4d1c101fc0 2024-04-25T13:04:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.984+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a0140 con 0x7f4d1c101fc0 2024-04-25T13:04:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.984+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a0690 con 0x7f4d1c101fc0 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d197fa640 1 -- 172.21.15.38:0/3237039028 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4d0c00fab0 con 0x7f4d1c101fc0 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 Processor -- start 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 -- start start 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d197fa640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4d040a3bb0 0x7f4d040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf5a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5ab20 0x560e2cf613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e2ceb9c50 con 0x560e2cf58f90 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e2ce56830 con 0x560e2cf5a160 2024-04-25T13:04:13.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d25394740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560e2ceb5c10 con 0x560e2cf5ab20 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d197fa640 1 -- 172.21.15.38:0/3237039028 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4d0c0d07d0 con 0x7f4d1c101fc0 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.986+0000 7f4d20b38640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4d040a3bb0 0x7f4d040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41030/0 (socket says 172.21.15.38:41030) 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3739696654 learned_addr learned my addr 172.21.15.38:0/3739696654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1b7fe640 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf5a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3739696654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5ab20 msgr2=0x560e2cf613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5ab20 0x560e2cf613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3739696654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 msgr2=0x560e2cf5a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf5a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 -- 172.21.15.38:0/3739696654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2cbcc2d0 con 0x560e2cf58f90 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d20b38640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4d040a3bb0 0x7f4d040a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d14004af0 tx=0x7f4d14009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf593b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c099510 tx=0x7f4d0c09c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8729726447869317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d027fc640 1 -- 172.21.15.38:0/3739696654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c013860 con 0x560e2cf58f90 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d027fc640 1 -- 172.21.15.38:0/3739696654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4d0c01c660 con 0x560e2cf58f90 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.987+0000 7f4d027fc640 1 -- 172.21.15.38:0/3739696654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c01c3b0 con 0x560e2cf58f90 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 -- 172.21.15.38:0/3739696654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 msgr2=0x560e2cf593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf593b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c099510 tx=0x7f4d0c09c560 comp rx=0 tx=0).stop 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 -- 172.21.15.38:0/3739696654 shutdown_connections 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5ab20 0x560e2cf613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf5a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 --2- 172.21.15.38:0/3739696654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf593b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 -- 172.21.15.38:0/3739696654 >> 172.21.15.38:0/3739696654 conn(0x560e2cf50070 msgr2=0x560e2cf50470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 -- 172.21.15.38:0/3739696654 shutdown_connections 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 -- 172.21.15.38:0/3739696654 wait complete. 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 -- start start 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.988+0000 7f4d25394740 1 Processor -- start 2024-04-25T13:04:13.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf6aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf61b30 con 0x560e2cf58f90 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e2ce54470 con 0x560e2cf5ab20 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf61cb0 con 0x560e2cf5a160 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 10 client.0 _ll_register_callbacks cb 0x560e2ce79f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf634d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41036/0 (socket says 172.21.15.38:41036) 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 -- 172.21.15.38:0/193475029 learned_addr learned my addr 172.21.15.38:0/193475029 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d20b38640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf6aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1b7fe640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 -- 172.21.15.38:0/193475029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 msgr2=0x560e2cf63a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf63a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 -- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 msgr2=0x560e2cf6aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf6aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 -- 172.21.15.38:0/193475029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2cc966b0 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d20b38640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf634d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0a51f0 tx=0x7f4d0c01df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d2d1d3e85338f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/193475029 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0a0ef0 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/193475029 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4d0c018b90 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.989+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0c01c660 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/193475029 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d0c01c3b0 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e2cf6ddc0 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/193475029 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4d0c01c660 con 0x560e2cf58f90 2024-04-25T13:04:13.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/193475029 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d0c0a30f0 con 0x560e2cf58f90 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 20 client.15081 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cf6f110 0x560e2cf712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 4 client.15081 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 --> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e2cec7670 con 0x560e2cf6f110 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.990+0000 7f4d1b7fe640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cf6f110 0x560e2cf712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.991+0000 7f4d1b7fe640 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cf6f110 0x560e2cf712b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65207a45d9f69890 server_cookie=4f082ea0883af38f in_seq=0 out_seq=0 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.991+0000 7f4cfb7fe640 10 client.15081 ms_handle_connect on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.991+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/193475029 <== mds.2 v2:172.21.15.104:6836/2247876378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e2cec7670 con 0x560e2cf6f110 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.991+0000 7f4cfb7fe640 10 client.15081 handle_command_reply: tid=0 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.991+0000 7f4d25394740 1 client.15081 shutdown 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 msgr2=0x560e2cf634d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf634d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0a51f0 tx=0x7f4d0c01df20 comp rx=0 tx=0).stop 2024-04-25T13:04:13.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 shutdown_connections 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cf6f110 0x560e2cf712b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4cfb7fe640 0 client.15081 ms_handle_reset on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 --2- 172.21.15.38:0/193475029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf634d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 >> 172.21.15.38:0/193475029 conn(0x560e2cf50070 msgr2=0x560e2cf4f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 shutdown_connections 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 1 -- 172.21.15.38:0/193475029 wait complete. 2024-04-25T13:04:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:13.992+0000 7f4d25394740 20 client.15081 trim_cache size 0 max 16384 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 Processor -- start 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 -- start start 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf4b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf6f5f0 con 0x560e2cf58f90 2024-04-25T13:04:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf6fcc0 con 0x560e2cf5ab20 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d25394740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf70390 con 0x560e2cf5a160 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf4b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf4b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41044/0 (socket says 172.21.15.38:41044) 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 -- 172.21.15.38:0/539380240 learned_addr learned my addr 172.21.15.38:0/539380240 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 -- 172.21.15.38:0/539380240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 msgr2=0x560e2cf6a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 -- 172.21.15.38:0/539380240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 msgr2=0x560e2cf630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 -- 172.21.15.38:0/539380240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2cf50d40 con 0x560e2cf58f90 2024-04-25T13:04:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf4b0b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c099510 tx=0x7f4d0c09b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcbc893048ef98c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d01ffb640 1 -- 172.21.15.38:0/539380240 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0b5070 con 0x560e2cf58f90 2024-04-25T13:04:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d01ffb640 1 -- 172.21.15.38:0/539380240 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4d0c0a0ef0 con 0x560e2cf58f90 2024-04-25T13:04:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.011+0000 7f4d01ffb640 1 -- 172.21.15.38:0/539380240 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0a30f0 con 0x560e2cf58f90 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 -- 172.21.15.38:0/539380240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 msgr2=0x560e2cf4b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf4b0b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c099510 tx=0x7f4d0c09b050 comp rx=0 tx=0).stop 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 -- 172.21.15.38:0/539380240 shutdown_connections 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5ab20 0x560e2cf630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf5a160 0x560e2cf6a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 --2- 172.21.15.38:0/539380240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf58f90 0x560e2cf4b0b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 -- 172.21.15.38:0/539380240 >> 172.21.15.38:0/539380240 conn(0x560e2cf50070 msgr2=0x560e2cf5d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 -- 172.21.15.38:0/539380240 shutdown_connections 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 -- 172.21.15.38:0/539380240 wait complete. 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.012+0000 7f4d25394740 1 -- start start 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 Processor -- start 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf58f90 0x560e2cf66950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 0x560e2cf693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf71130 con 0x560e2cf5ab20 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf70a60 con 0x560e2cf5a160 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560e2cf71800 con 0x560e2cf58f90 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d25394740 10 client.0 _ll_register_callbacks cb 0x7f4d1000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 0x560e2cf693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 0x560e2cf693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41046/0 (socket says 172.21.15.38:41046) 2024-04-25T13:04:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 -- 172.21.15.38:0/1019584721 learned_addr learned my addr 172.21.15.38:0/1019584721 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d1b7fe640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf58f90 0x560e2cf66950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 -- 172.21.15.38:0/1019584721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf58f90 msgr2=0x560e2cf66950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf58f90 0x560e2cf66950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 -- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 msgr2=0x560e2cf68ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf68ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 -- 172.21.15.38:0/1019584721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2cf699a0 con 0x560e2cf5ab20 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf58f90 0x560e2cf66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4d20b38640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 0x560e2cf693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4d140319b0 tx=0x7f4d140095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3238595f5a3244d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.013+0000 7f4cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/1019584721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14004440 con 0x560e2cf5ab20 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/1019584721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4d140045e0 con 0x560e2cf5ab20 2024-04-25T13:04:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e2cec14e0 con 0x560e2cf5ab20 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/1019584721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d140047e0 con 0x560e2cf5ab20 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e2cf6aca0 con 0x560e2cf5ab20 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/1019584721 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x560e2cec14e0 con 0x560e2cf5ab20 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/1019584721 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d14030a80 con 0x560e2cf5ab20 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 10 client.15087 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 20 client.15087 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cfda820 0x560e2cfd9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 4 client.15087 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 --> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] -- command(tid 0: {"prefix": "status"}) -- 0x560e2ce3d9a0 con 0x560e2cfda820 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.014+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cfda820 0x560e2cfd9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d1bfff640 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cfda820 0x560e2cfd9530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85351029a89e3cad server_cookie=ff25c11c92bf813d in_seq=0 out_seq=0 2024-04-25T13:04:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4cfb7fe640 10 client.15087 ms_handle_connect on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4cfb7fe640 1 -- 172.21.15.38:0/1019584721 <== mds.2 v2:172.21.15.104:6836/2247876378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x560e2ce3d9a0 con 0x560e2cfda820 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4cfb7fe640 10 client.15087 handle_command_reply: tid=0 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 client.15087 shutdown 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 msgr2=0x560e2cf693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 0x560e2cf693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4d140319b0 tx=0x7f4d140095c0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 shutdown_connections 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x560e2cfda820 0x560e2cfd9530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e2cf5ab20 0x560e2cf693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4cfb7fe640 0 client.15087 ms_handle_reset on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e2cf5a160 0x560e2cf68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 --2- 172.21.15.38:0/1019584721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e2cf58f90 0x560e2cf66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 >> 172.21.15.38:0/1019584721 conn(0x560e2cf50070 msgr2=0x560e2cf630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 shutdown_connections 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 1 -- 172.21.15.38:0/1019584721 wait complete. 2024-04-25T13:04:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.015+0000 7f4d25394740 20 client.15087 trim_cache size 0 max 16384 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24482, 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 85.075033227000006, 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:14.021 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:14.022 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:14.022 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 104.76123049 2024-04-25T13:04:14.022 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4d040a3bb0 msgr2=0x7f4d040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4d040a3bb0 0x7f4d040a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d14004af0 tx=0x7f4d14009800 comp rx=0 tx=0).stop 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 msgr2=0x7f4d1c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c970 tx=0x7f4d0c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 shutdown_connections 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4d040a3bb0 0x7f4d040a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 --2- 172.21.15.38:0/3237039028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198230 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 >> 172.21.15.38:0/3237039028 conn(0x7f4d1c0fd530 msgr2=0x7f4d1c1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 shutdown_connections 2024-04-25T13:04:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.016+0000 7f4d2259b640 1 -- 172.21.15.38:0/3237039028 wait complete. 2024-04-25T13:04:14.025 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 -- 172.21.15.38:0/1022291497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca8103b70 msgr2=0x7fdca810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 --2- 172.21.15.38:0/1022291497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca8103b70 0x7fdca810a400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009ca0 tx=0x7fdca002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 -- 172.21.15.38:0/1022291497 shutdown_connections 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 --2- 172.21.15.38:0/1022291497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca8103b70 0x7fdca810a400 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 --2- 172.21.15.38:0/1022291497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca81031b0 0x7fdca8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 --2- 172.21.15.38:0/1022291497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca8101fc0 0x7fdca81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 -- 172.21.15.38:0/1022291497 >> 172.21.15.38:0/1022291497 conn(0x7fdca80fd530 msgr2=0x7fdca80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 -- 172.21.15.38:0/1022291497 shutdown_connections 2024-04-25T13:04:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.131+0000 7fdcb0243640 1 -- 172.21.15.38:0/1022291497 wait complete. 2024-04-25T13:04:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 Processor -- start 2024-04-25T13:04:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 -- start start 2024-04-25T13:04:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca8101fc0 0x7fdca8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 0x7fdca8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca8103b70 0x7fdca819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810dc20 con 0x7fdca81031b0 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810daa0 con 0x7fdca8101fc0 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcb0243640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810dda0 con 0x7fdca8103b70 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 0x7fdca8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 0x7fdca8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41066/0 (socket says 172.21.15.38:41066) 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 -- 172.21.15.38:0/359660442 learned_addr learned my addr 172.21.15.38:0/359660442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca8101fc0 0x7fdca8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 -- 172.21.15.38:0/359660442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca8103b70 msgr2=0x7fdca819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca8103b70 0x7fdca819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 -- 172.21.15.38:0/359660442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca8101fc0 msgr2=0x7fdca8198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca8101fc0 0x7fdca8198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 -- 172.21.15.38:0/359660442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca819fe70 con 0x7fdca81031b0 2024-04-25T13:04:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.132+0000 7fdcad7de640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 0x7fdca8198790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d770 tx=0x7fdc9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.133+0000 7fdc9effd640 1 -- 172.21.15.38:0/359660442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98017070 con 0x7fdca81031b0 2024-04-25T13:04:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.133+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca81a0160 con 0x7fdca81031b0 2024-04-25T13:04:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.133+0000 7fdc9effd640 1 -- 172.21.15.38:0/359660442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdc980054f0 con 0x7fdca81031b0 2024-04-25T13:04:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.133+0000 7fdc9effd640 1 -- 172.21.15.38:0/359660442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98004de0 con 0x7fdca81031b0 2024-04-25T13:04:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.133+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca81a06b0 con 0x7fdca81031b0 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdc9effd640 1 -- 172.21.15.38:0/359660442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdc98004f80 con 0x7fdca81031b0 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 Processor -- start 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 -- start start 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6cd50 0x55fa09e6d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6d710 0x55fa09e73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa09dd0c50 con 0x55fa09e6bb80 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa09d6d830 con 0x55fa09e6d710 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.135+0000 7fdcb303a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa09dccc10 con 0x55fa09e6cd50 2024-04-25T13:04:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e6bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41070/0 (socket says 172.21.15.38:41070) 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2012131727 learned_addr learned my addr 172.21.15.38:0/2012131727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdc9effd640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc940a3bb0 0x7fdc940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6cd50 0x55fa09e6d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcad7de640 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6d710 0x55fa09e73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2012131727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6cd50 msgr2=0x55fa09e6d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6cd50 0x55fa09e6d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2012131727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6d710 msgr2=0x55fa09e73fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6d710 0x55fa09e73fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2012131727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa09ae32d0 con 0x55fa09e6bb80 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcad7de640 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6d710 0x55fa09e73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdc9effd640 1 -- 172.21.15.38:0/359660442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdc980cff50 con 0x7fdca81031b0 2024-04-25T13:04:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc940a3bb0 0x7fdc940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e6bfa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdca0004af0 tx=0x7fdca00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c8a491c238ae6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdc93fff640 1 -- 172.21.15.38:0/2012131727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca004d070 con 0x55fa09e6bb80 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdc93fff640 1 -- 172.21.15.38:0/2012131727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdca0036040 con 0x55fa09e6bb80 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.136+0000 7fdc93fff640 1 -- 172.21.15.38:0/2012131727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca00305c0 con 0x55fa09e6bb80 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 -- 172.21.15.38:0/2012131727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 msgr2=0x55fa09e6bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc940a3bb0 0x7fdc940a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdca0046000 tx=0x7fdca004b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e6bfa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdca0004af0 tx=0x7fdca00098d0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 -- 172.21.15.38:0/2012131727 shutdown_connections 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6d710 0x55fa09e73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6cd50 0x55fa09e6d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2012131727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e6bfa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 -- 172.21.15.38:0/2012131727 >> 172.21.15.38:0/2012131727 conn(0x55fa09e67070 msgr2=0x55fa09e67470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 -- 172.21.15.38:0/2012131727 shutdown_connections 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 -- 172.21.15.38:0/2012131727 wait complete. 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.137+0000 7fdcb303a740 1 -- start start 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 Processor -- start 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e75e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e7d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e745a0 con 0x55fa09e6d710 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa09d6b470 con 0x55fa09e6cd50 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e74720 con 0x55fa09e6bb80 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcb303a740 10 client.0 _ll_register_callbacks cb 0x55fa09d90f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e7d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e7d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41078/0 (socket says 172.21.15.38:41078) 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2805824054 learned_addr learned my addr 172.21.15.38:0/2805824054 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e75e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 msgr2=0x55fa09e75e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcad7de640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e75e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2805824054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 msgr2=0x55fa09e763c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e763c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2805824054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa09bad6b0 con 0x55fa09e6d710 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e75e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.138+0000 7fdcad7de640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e7d400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdca0005c90 tx=0x7fdca0009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6cdaec097f1239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdc917fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdc917fa640 1 -- 172.21.15.38:0/2805824054 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca004d030 con 0x55fa09e6d710 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdc917fa640 1 -- 172.21.15.38:0/2805824054 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdca0004490 con 0x55fa09e6d710 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca0036040 con 0x55fa09e6d710 2024-04-25T13:04:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdc917fa640 1 -- 172.21.15.38:0/2805824054 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdca002f310 con 0x55fa09e6d710 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa09e745a0 con 0x55fa09e6d710 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdc917fa640 1 -- 172.21.15.38:0/2805824054 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdca0036040 con 0x55fa09e6d710 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdc917fa640 1 -- 172.21.15.38:0/2805824054 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdca0044040 con 0x55fa09e6d710 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 20 client.15099 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09e82a40 0x55fa09e83eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 4 client.15099 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.139+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 --> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa09dde670 con 0x55fa09e82a40 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.140+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09e82a40 0x55fa09e83eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.140+0000 7fdc917fa640 10 client.15099 ms_handle_connect on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.140+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09e82a40 0x55fa09e83eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ffadf78fee1e335 server_cookie=fba15cebcf357477 in_seq=0 out_seq=0 2024-04-25T13:04:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdc917fa640 1 -- 172.21.15.38:0/2805824054 <== mds.1 v2:172.21.15.178:6838/391098246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa09dde670 con 0x55fa09e82a40 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdc917fa640 10 client.15099 handle_command_reply: tid=0 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 client.15099 shutdown 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 msgr2=0x55fa09e7d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e7d400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdca0005c90 tx=0x7fdca0009940 comp rx=0 tx=0).stop 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 shutdown_connections 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e7d400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09e82a40 0x55fa09e83eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2805824054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e75e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdc917fa640 0 client.15099 ms_handle_reset on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 >> 172.21.15.38:0/2805824054 conn(0x55fa09e67070 msgr2=0x55fa09e661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 shutdown_connections 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 1 -- 172.21.15.38:0/2805824054 wait complete. 2024-04-25T13:04:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.141+0000 7fdcb303a740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.159+0000 7fdcb303a740 1 Processor -- start 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.159+0000 7fdcb303a740 1 -- start start 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e74ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e7d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e70870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcb303a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e821f0 con 0x55fa09e6d710 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcb303a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e828c0 con 0x55fa09e6cd50 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcb303a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e82f90 con 0x55fa09e6bb80 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e70870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e70870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41092/0 (socket says 172.21.15.38:41092) 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2315907239 learned_addr learned my addr 172.21.15.38:0/2315907239 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e74ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcad7de640 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e7d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2315907239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 msgr2=0x55fa09e74ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e74ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2315907239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 msgr2=0x55fa09e7d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e7d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 -- 172.21.15.38:0/2315907239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa09e74830 con 0x55fa09e6d710 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e74ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e70870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009ca0 tx=0x7fdca002f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ed7deddc19a99e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdc937fe640 1 -- 172.21.15.38:0/2315907239 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca004d040 con 0x55fa09e6d710 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdc937fe640 1 -- 172.21.15.38:0/2315907239 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdca0036040 con 0x55fa09e6d710 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.160+0000 7fdc937fe640 1 -- 172.21.15.38:0/2315907239 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca00309a0 con 0x55fa09e6d710 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 -- 172.21.15.38:0/2315907239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 msgr2=0x55fa09e70870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e70870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009ca0 tx=0x7fdca002f4b0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 -- 172.21.15.38:0/2315907239 shutdown_connections 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6d710 0x55fa09e70870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e7d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 --2- 172.21.15.38:0/2315907239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6bb80 0x55fa09e74ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 -- 172.21.15.38:0/2315907239 >> 172.21.15.38:0/2315907239 conn(0x55fa09e67070 msgr2=0x55fa09e620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 -- 172.21.15.38:0/2315907239 shutdown_connections 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 -- 172.21.15.38:0/2315907239 wait complete. 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 -- start start 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.161+0000 7fdcb303a740 1 Processor -- start 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e78b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6d710 0x55fa09e7bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e83d30 con 0x55fa09e6bb80 2024-04-25T13:04:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e83660 con 0x55fa09e6cd50 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa09e84400 con 0x55fa09e6d710 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 10 client.0 _ll_register_callbacks cb 0x7fdca004b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e785d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41094/0 (socket says 172.21.15.38:41094) 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 -- 172.21.15.38:0/401034581 learned_addr learned my addr 172.21.15.38:0/401034581 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcae7e0640 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6d710 0x55fa09e7bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 -- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6d710 msgr2=0x55fa09e7bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6d710 0x55fa09e7bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 -- 172.21.15.38:0/401034581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 msgr2=0x55fa09e78b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e78b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 -- 172.21.15.38:0/401034581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa09e7c0d0 con 0x55fa09e6bb80 2024-04-25T13:04:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcadfdf640 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e785d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdca400dc30 tx=0x7fdca4019f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c034ff82af71fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdc917fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdc917fa640 1 -- 172.21.15.38:0/401034581 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4019290 con 0x55fa09e6bb80 2024-04-25T13:04:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdc917fa640 1 -- 172.21.15.38:0/401034581 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdca4019430 con 0x55fa09e6bb80 2024-04-25T13:04:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.162+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca0036040 con 0x55fa09e6bb80 2024-04-25T13:04:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdc917fa640 1 -- 172.21.15.38:0/401034581 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdca4019630 con 0x55fa09e6bb80 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa09e7d4a0 con 0x55fa09e6bb80 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdc917fa640 1 -- 172.21.15.38:0/401034581 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdca0036040 con 0x55fa09e6bb80 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdc917fa640 1 -- 172.21.15.38:0/401034581 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdca402d400 con 0x55fa09e6bb80 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 20 client.15105 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09eed280 0x55fa09eebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 4 client.15105 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 --> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] -- command(tid 0: {"prefix": "status"}) -- 0x55fa09dd84e0 con 0x55fa09eed280 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.163+0000 7fdcad7de640 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09eed280 0x55fa09eebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdc917fa640 10 client.15105 ms_handle_connect on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcad7de640 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09eed280 0x55fa09eebf90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94c2995294b7ed7b server_cookie=b4fea4e51fffa00a in_seq=0 out_seq=0 2024-04-25T13:04:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdc917fa640 1 -- 172.21.15.38:0/401034581 <== mds.1 v2:172.21.15.178:6838/391098246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fa09dd84e0 con 0x55fa09eed280 2024-04-25T13:04:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdc917fa640 10 client.15105 handle_command_reply: tid=0 2024-04-25T13:04:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 client.15105 shutdown 2024-04-25T13:04:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 msgr2=0x55fa09e785d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e785d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdca400dc30 tx=0x7fdca4019f90 comp rx=0 tx=0).stop 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 shutdown_connections 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa09e6d710 0x55fa09e7bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x55fa09eed280 0x55fa09eebf90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa09e6cd50 0x55fa09e78b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 --2- 172.21.15.38:0/401034581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa09e6bb80 0x55fa09e785d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdc917fa640 0 client.15105 ms_handle_reset on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 >> 172.21.15.38:0/401034581 conn(0x55fa09e67070 msgr2=0x55fa09e74ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 shutdown_connections 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 1 -- 172.21.15.38:0/401034581 wait complete. 2024-04-25T13:04:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.164+0000 7fdcb303a740 20 client.15105 trim_cache size 0 max 16384 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24484, 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 86.235418233000004, 2024-04-25T13:04:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:14.173 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:14.173 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:14.173 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 102.080754072 2024-04-25T13:04:14.173 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc940a3bb0 msgr2=0x7fdc940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc940a3bb0 0x7fdc940a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdca0046000 tx=0x7fdca004b000 comp rx=0 tx=0).stop 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 msgr2=0x7fdca8198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 0x7fdca8198790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d770 tx=0x7fdc9800dc30 comp rx=0 tx=0).stop 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 shutdown_connections 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdc940a3bb0 0x7fdc940a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca8103b70 0x7fdca819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdca81031b0 0x7fdca8198790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 --2- 172.21.15.38:0/359660442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca8101fc0 0x7fdca8198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 >> 172.21.15.38:0/359660442 conn(0x7fdca80fd530 msgr2=0x7fdca81063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 shutdown_connections 2024-04-25T13:04:14.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.165+0000 7fdcb0243640 1 -- 172.21.15.38:0/359660442 wait complete. 2024-04-25T13:04:14.175 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:14 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2970146678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:14.175 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:14 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/961199512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:14.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 -- 172.21.15.38:0/4045824416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 msgr2=0x7f8448101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 --2- 172.21.15.38:0/4045824416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448101280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8438009900 tx=0x7f843802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 -- 172.21.15.38:0/4045824416 shutdown_connections 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 --2- 172.21.15.38:0/4045824416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8448102a00 0x7f8448109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 --2- 172.21.15.38:0/4045824416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102040 0x7f84481024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 --2- 172.21.15.38:0/4045824416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448101280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 -- 172.21.15.38:0/4045824416 >> 172.21.15.38:0/4045824416 conn(0x7f84480fc440 msgr2=0x7f84480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 -- 172.21.15.38:0/4045824416 shutdown_connections 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.280+0000 7f844cd37640 1 -- 172.21.15.38:0/4045824416 wait complete. 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 Processor -- start 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 -- start start 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8448102040 0x7f8448198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102a00 0x7f844819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f844810cb20 con 0x7f8448100e60 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f844810c9a0 con 0x7f8448102a00 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844cd37640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f844810cca0 con 0x7f8448102040 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41122/0 (socket says 172.21.15.38:41122) 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f844659c640 1 -- 172.21.15.38:0/277876190 learned_addr learned my addr 172.21.15.38:0/277876190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f8445d9b640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8448102040 0x7f8448198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.281+0000 7f8446d9d640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102a00 0x7f844819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844659c640 1 -- 172.21.15.38:0/277876190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8448102040 msgr2=0x7f8448198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844659c640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8448102040 0x7f8448198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844659c640 1 -- 172.21.15.38:0/277876190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102a00 msgr2=0x7f844819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844659c640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102a00 0x7f844819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844659c640 1 -- 172.21.15.38:0/277876190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844819fe40 con 0x7f8448100e60 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f8446d9d640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102a00 0x7f844819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844659c640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448198220 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8438006010 tx=0x7f84380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f84377fe640 1 -- 172.21.15.38:0/277876190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843803f030 con 0x7f8448100e60 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84481a00d0 con 0x7f8448100e60 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f84377fe640 1 -- 172.21.15.38:0/277876190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8438003cd0 con 0x7f8448100e60 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f84377fe640 1 -- 172.21.15.38:0/277876190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438035070 con 0x7f8448100e60 2024-04-25T13:04:14.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.282+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84481a05c0 con 0x7f8448100e60 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.284+0000 7f84377fe640 1 -- 172.21.15.38:0/277876190 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8438042030 con 0x7f8448100e60 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.284+0000 7f844fa50740 1 Processor -- start 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.284+0000 7f844fa50740 1 -- start start 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844fa50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a110578ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844fa50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110579cd0 0x55a11057a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844fa50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a11057af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844fa50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a1104ddc50 con 0x55a110578ab0 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844fa50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a11047a830 con 0x55a11057a670 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844fa50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1104d9c10 con 0x55a110579cd0 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a110578ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a110578ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41124/0 (socket says 172.21.15.38:41124) 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 -- 172.21.15.38:0/3604693253 learned_addr learned my addr 172.21.15.38:0/3604693253 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f844659c640 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110579cd0 0x55a11057a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8445d9b640 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a11057af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f84377fe640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f84300a3bb0 0x7f84300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 -- 172.21.15.38:0/3604693253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110579cd0 msgr2=0x55a11057a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110579cd0 0x55a11057a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 -- 172.21.15.38:0/3604693253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 msgr2=0x55a11057af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a11057af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 -- 172.21.15.38:0/3604693253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1101f02d0 con 0x55a110578ab0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8445d9b640 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a11057af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f84300a3bb0 0x7f84300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f84377fe640 1 -- 172.21.15.38:0/277876190 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f84380f0b90 con 0x7f8448100e60 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.285+0000 7f8446d9d640 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a110578ed0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8440008f60 tx=0x7f8440008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca35dc560389e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f8434ff9640 1 -- 172.21.15.38:0/3604693253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844001a8e0 con 0x55a110578ab0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f8446d9d640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f84300a3bb0 0x7f84300a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f844001e000 tx=0x7f844001f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f8434ff9640 1 -- 172.21.15.38:0/3604693253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8440019410 con 0x55a110578ab0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f8434ff9640 1 -- 172.21.15.38:0/3604693253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844000e610 con 0x55a110578ab0 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 -- 172.21.15.38:0/3604693253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 msgr2=0x55a110578ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a110578ed0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8440008f60 tx=0x7f8440008f90 comp rx=0 tx=0).stop 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 -- 172.21.15.38:0/3604693253 shutdown_connections 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a11057af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110579cd0 0x55a11057a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 --2- 172.21.15.38:0/3604693253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a110578ed0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.286+0000 7f844fa50740 1 -- 172.21.15.38:0/3604693253 >> 172.21.15.38:0/3604693253 conn(0x55a110574070 msgr2=0x55a110574470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 -- 172.21.15.38:0/3604693253 shutdown_connections 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 -- 172.21.15.38:0/3604693253 wait complete. 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 -- start start 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 Processor -- start 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110578ab0 0x55a11058b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a11058bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a110592d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a11057b690 con 0x55a110579cd0 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a110478470 con 0x55a110578ab0 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a11057b810 con 0x55a11057a670 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844fa50740 10 client.0 _ll_register_callbacks cb 0x55a11049df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f8445d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a11058bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f8445d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a11058bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41134/0 (socket says 172.21.15.38:41134) 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f8445d9b640 1 -- 172.21.15.38:0/3297823809 learned_addr learned my addr 172.21.15.38:0/3297823809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f844659c640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110578ab0 0x55a11058b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.287+0000 7f8446d9d640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a110592d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8445d9b640 1 -- 172.21.15.38:0/3297823809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 msgr2=0x55a110592d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8445d9b640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a110592d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8445d9b640 1 -- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110578ab0 msgr2=0x55a11058b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8445d9b640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110578ab0 0x55a11058b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8445d9b640 1 -- 172.21.15.38:0/3297823809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1102ba6b0 con 0x55a110579cd0 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8446d9d640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a110592d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f8445d9b640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a11058bd20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f843c00dfb0 tx=0x7f843c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4020655c1d3ca8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f842dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f842dffb640 1 -- 172.21.15.38:0/3297823809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005220 con 0x55a110579cd0 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f842dffb640 1 -- 172.21.15.38:0/3297823809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f843c0053c0 con 0x55a110579cd0 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8440019410 con 0x55a110579cd0 2024-04-25T13:04:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f842dffb640 1 -- 172.21.15.38:0/3297823809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f843c0055c0 con 0x55a110579cd0 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1105961a0 con 0x55a110579cd0 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f842dffb640 1 -- 172.21.15.38:0/3297823809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8440019410 con 0x55a110579cd0 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f842dffb640 1 -- 172.21.15.38:0/3297823809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f843c012a50 con 0x55a110579cd0 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 20 client.15117 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1105988c0 0x55a110599d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 4 client.15117 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.288+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 --> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1104eb670 con 0x55a1105988c0 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.289+0000 7f844659c640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1105988c0 0x55a110599d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.289+0000 7f844659c640 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1105988c0 0x55a110599d30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33e44310ce88465 server_cookie=a7568de3a9a32719 in_seq=0 out_seq=0 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.289+0000 7f842dffb640 10 client.15117 ms_handle_connect on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f842dffb640 1 -- 172.21.15.38:0/3297823809 <== mds.0 v2:172.21.15.104:6838/1961595450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1104eb670 con 0x55a1105988c0 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f842dffb640 10 client.15117 handle_command_reply: tid=0 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 client.15117 shutdown 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 msgr2=0x55a11058bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a11058bd20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f843c00dfb0 tx=0x7f843c011870 comp rx=0 tx=0).stop 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 shutdown_connections 2024-04-25T13:04:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1105988c0 0x55a110599d30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a110592d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a11058bd20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f842dffb640 0 client.15117 ms_handle_reset on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 --2- 172.21.15.38:0/3297823809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110578ab0 0x55a11058b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 >> 172.21.15.38:0/3297823809 conn(0x55a110574070 msgr2=0x55a1105770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 shutdown_connections 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 1 -- 172.21.15.38:0/3297823809 wait complete. 2024-04-25T13:04:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.290+0000 7f844fa50740 20 client.15117 trim_cache size 0 max 16384 2024-04-25T13:04:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 Processor -- start 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 -- start start 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a11057bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a11057c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a110598070 con 0x55a110578ab0 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a110598740 con 0x55a110579cd0 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844fa50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a110598e10 con 0x55a11057a670 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a11057bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a11057bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41148/0 (socket says 172.21.15.38:41148) 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.308+0000 7f844659c640 1 -- 172.21.15.38:0/2642878233 learned_addr learned my addr 172.21.15.38:0/2642878233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f8445d9b640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844659c640 1 -- 172.21.15.38:0/2642878233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 msgr2=0x55a11057c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f8446d9d640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a11057c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844659c640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a11057c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844659c640 1 -- 172.21.15.38:0/2642878233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110579cd0 msgr2=0x55a110592d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844659c640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844659c640 1 -- 172.21.15.38:0/2642878233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a110589ae0 con 0x55a110578ab0 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f8446d9d640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a11057c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f8445d9b640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844659c640 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a11057bbe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f84380b9800 tx=0x7f84380ee520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc47dce1b302d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f842ffff640 1 -- 172.21.15.38:0/2642878233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380f6070 con 0x55a110578ab0 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f842ffff640 1 -- 172.21.15.38:0/2642878233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f84380bce30 con 0x55a110578ab0 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f842ffff640 1 -- 172.21.15.38:0/2642878233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438049040 con 0x55a110578ab0 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844fa50740 1 -- 172.21.15.38:0/2642878233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 msgr2=0x55a11057bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.309+0000 7f844fa50740 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a11057bbe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f84380b9800 tx=0x7f84380ee520 comp rx=0 tx=0).stop 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- 172.21.15.38:0/2642878233 shutdown_connections 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11057a670 0x55a11057c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 --2- 172.21.15.38:0/2642878233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110578ab0 0x55a11057bbe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- 172.21.15.38:0/2642878233 >> 172.21.15.38:0/2642878233 conn(0x55a110574070 msgr2=0x55a110589e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- 172.21.15.38:0/2642878233 shutdown_connections 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- 172.21.15.38:0/2642878233 wait complete. 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- start start 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 Processor -- start 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110578ab0 0x55a110587800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a110587d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a110599bb0 con 0x55a110579cd0 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a1105994e0 con 0x55a11057a670 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a11059a280 con 0x55a110578ab0 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844fa50740 10 client.0 _ll_register_callbacks cb 0x7f843c00d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f8445d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f8445d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41150/0 (socket says 172.21.15.38:41150) 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f8445d9b640 1 -- 172.21.15.38:0/932276919 learned_addr learned my addr 172.21.15.38:0/932276919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f844659c640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110578ab0 0x55a110587800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f8446d9d640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a110587d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f8445d9b640 1 -- 172.21.15.38:0/932276919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110578ab0 msgr2=0x55a110587800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.310+0000 7f8445d9b640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110578ab0 0x55a110587800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f8445d9b640 1 -- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 msgr2=0x55a110587d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f8445d9b640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a110587d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f8445d9b640 1 -- 172.21.15.38:0/932276919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a110588430 con 0x55a110579cd0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f8445d9b640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a110592d20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f843c012ec0 tx=0x7f843c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3053645524733470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f842dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f842dffb640 1 -- 172.21.15.38:0/932276919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c034020 con 0x55a110579cd0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f842dffb640 1 -- 172.21.15.38:0/932276919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f843c004220 con 0x55a110579cd0 2024-04-25T13:04:14.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84380bce30 con 0x55a110579cd0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f842dffb640 1 -- 172.21.15.38:0/932276919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f843c02b050 con 0x55a110579cd0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1105fe870 con 0x55a110579cd0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f842dffb640 1 -- 172.21.15.38:0/932276919 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f84380bce30 con 0x55a110579cd0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f842dffb640 1 -- 172.21.15.38:0/932276919 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f843c039070 con 0x55a110579cd0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 20 client.15123 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1106033e0 0x55a1106020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 4 client.15123 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.311+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 --> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] -- command(tid 0: {"prefix": "status"}) -- 0x55a1104e54e0 con 0x55a1106033e0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844659c640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1106033e0 0x55a1106020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844659c640 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1106033e0 0x55a1106020f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191816b314948422 server_cookie=7ea35d5d34019fac in_seq=0 out_seq=0 2024-04-25T13:04:14.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f842dffb640 10 client.15123 ms_handle_connect on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f842dffb640 1 -- 172.21.15.38:0/932276919 <== mds.0 v2:172.21.15.104:6838/1961595450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a1104e54e0 con 0x55a1106033e0 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f842dffb640 10 client.15123 handle_command_reply: tid=0 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 client.15123 shutdown 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 msgr2=0x55a110592d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a110592d20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f843c012ec0 tx=0x7f843c00d890 comp rx=0 tx=0).stop 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 shutdown_connections 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a11057a670 0x55a110587d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a110579cd0 0x55a110592d20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x55a1106033e0 0x55a1106020f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f842dffb640 0 client.15123 ms_handle_reset on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 --2- 172.21.15.38:0/932276919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a110578ab0 0x55a110587800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 >> 172.21.15.38:0/932276919 conn(0x55a110574070 msgr2=0x55a11057bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 shutdown_connections 2024-04-25T13:04:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.312+0000 7f844fa50740 1 -- 172.21.15.38:0/932276919 wait complete. 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844fa50740 20 client.15123 trim_cache size 0 max 16384 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24518, 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 89.419003728000007, 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:14.318 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:14.319 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:14.319 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 93.669644363000003 2024-04-25T13:04:14.319 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f84300a3bb0 msgr2=0x7f84300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f84300a3bb0 0x7f84300a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f844001e000 tx=0x7f844001f000 comp rx=0 tx=0).stop 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 msgr2=0x7f8448198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448198220 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8438006010 tx=0x7f84380042f0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 shutdown_connections 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f84300a3bb0 0x7f84300a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8448102a00 0x7f844819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8448102040 0x7f8448198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 --2- 172.21.15.38:0/277876190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448100e60 0x7f8448198220 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:14.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 >> 172.21.15.38:0/277876190 conn(0x7f84480fc440 msgr2=0x7f84480fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:14.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 shutdown_connections 2024-04-25T13:04:14.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:14.313+0000 7f844cd37640 1 -- 172.21.15.38:0/277876190 wait complete. 2024-04-25T13:04:14.322 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:04:14.323 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:04:14.332 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:14.332 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:04:14.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:14 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2970146678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:14.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:14 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/961199512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:14.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:14 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2970146678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:14.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:14 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/961199512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:14.576 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:15.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:15 smithi038 ceph-mon[29400]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:15.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:15 smithi178 ceph-mon[33233]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:15.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:15 smithi104 ceph-mon[32726]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:16.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/2597404797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 msgr2=0x7f1ec8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/2597404797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec8109df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000b0d0 tx=0x7f1ec000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/2597404797 shutdown_connections 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/2597404797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 0x7f1ec8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/2597404797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec8109df0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/2597404797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec8102ba0 0x7f1ec8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/2597404797 >> 172.21.15.38:0/2597404797 conn(0x7f1ec80fcf40 msgr2=0x7f1ec80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/2597404797 shutdown_connections 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.400+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/2597404797 wait complete. 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 Processor -- start 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 -- start start 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 0x7f1ec806b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec8102ba0 0x7f1ec819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec81a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810d580 con 0x7f1ec8103540 2024-04-25T13:04:16.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810d400 con 0x7f1ec81019a0 2024-04-25T13:04:16.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ecd1ea700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810d700 con 0x7f1ec8102ba0 2024-04-25T13:04:16.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ec759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec81a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ec6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 0x7f1ec806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.401+0000 7f1ec759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec81a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41166/0 (socket says 172.21.15.38:41166) 2024-04-25T13:04:16.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 -- 172.21.15.38:0/470025383 learned_addr learned my addr 172.21.15.38:0/470025383 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:16.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec659c700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec8102ba0 0x7f1ec819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 -- 172.21.15.38:0/470025383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec8102ba0 msgr2=0x7f1ec819d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec8102ba0 0x7f1ec819d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 -- 172.21.15.38:0/470025383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 msgr2=0x7f1ec806b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 0x7f1ec806b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 -- 172.21.15.38:0/470025383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a0900 con 0x7f1ec8103540 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec6d9d700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 0x7f1ec806b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ec759e700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec81a0030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009cd0 tx=0x7f1ec0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000bd60 con 0x7f1ec8103540 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1ec0006040 con 0x7f1ec8103540 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec001b820 con 0x7f1ec8103540 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a0b90 con 0x7f1ec8103540 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.402+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a0fe0 con 0x7f1ec8103540 2024-04-25T13:04:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.403+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8069730 con 0x7f1ec8103540 2024-04-25T13:04:16.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.405+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1ec001d470 con 0x7f1ec8103540 2024-04-25T13:04:16.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.405+0000 7f1eaffff700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1eb00a3cc0 0x7f1eb00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:16.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.405+0000 7f1ec6d9d700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1eb00a3cc0 0x7f1eb00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:16.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.405+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1ec00d18a0 con 0x7f1ec8103540 2024-04-25T13:04:16.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.406+0000 7f1ec6d9d700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1eb00a3cc0 0x7f1eb00a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8102a00 tx=0x7f1eb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.413+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ec002e330 con 0x7f1ec8103540 2024-04-25T13:04:16.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.638+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1ec8062e80 con 0x7f1eb00a3cc0 2024-04-25T13:04:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.639+0000 7f1eaffff700 1 -- 172.21.15.38:0/470025383 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1ec8062e80 con 0x7f1eb00a3cc0 2024-04-25T13:04:16.638 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-04-25T13:04:16.638 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-04-25T13:04:16.638 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-04-25T13:04:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.641+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1eb00a3cc0 msgr2=0x7f1eb00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.641+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1eb00a3cc0 0x7f1eb00a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8102a00 tx=0x7f1eb800b410 comp rx=0 tx=0).stop 2024-04-25T13:04:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.641+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 msgr2=0x7f1ec81a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.641+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec81a0030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009cd0 tx=0x7f1ec0007d90 comp rx=0 tx=0).stop 2024-04-25T13:04:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 shutdown_connections 2024-04-25T13:04:16.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f1eb00a3cc0 0x7f1eb00a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec81019a0 0x7f1ec806b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec8103540 0x7f1ec81a0030 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 --2- 172.21.15.38:0/470025383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec8102ba0 0x7f1ec819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 >> 172.21.15.38:0/470025383 conn(0x7f1ec80fcf40 msgr2=0x7f1ec8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 shutdown_connections 2024-04-25T13:04:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:16.642+0000 7f1ecd1ea700 1 -- 172.21.15.38:0/470025383 wait complete. 2024-04-25T13:04:17.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:17 smithi038 ceph-mon[29400]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:17.414 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:04:17.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:17 smithi178 ceph-mon[33233]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:17.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:17 smithi104 ceph-mon[32726]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:17.665 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:18.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:18 smithi038 ceph-mon[29400]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:18 smithi178 ceph-mon[33233]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:18.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:18 smithi104 ceph-mon[32726]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:19.200 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:19 smithi038 ceph-mon[29400]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:19.389 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:19 smithi178 ceph-mon[33233]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.438+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1614387147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf501087b0 msgr2=0x7fdf5010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.438+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1614387147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf501087b0 0x7fdf5010ac20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b0a0 tx=0x7fdf4800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1614387147 shutdown_connections 2024-04-25T13:04:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1614387147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf501087b0 0x7fdf5010ac20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1614387147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf50103550 0x7fdf50105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1614387147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf50105e80 0x7fdf50108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1614387147 >> 172.21.15.38:0/1614387147 conn(0x7fdf500fcf60 msgr2=0x7fdf500ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1614387147 shutdown_connections 2024-04-25T13:04:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1614387147 wait complete. 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 Processor -- start 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 -- start start 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.439+0000 7fdf56ebe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf50103550 0x7fdf50102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf56ebe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 0x7fdf50101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf56ebe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf501087b0 0x7fdf50101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf56ebe700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010e6a0 con 0x7fdf50105e80 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf56ebe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010e520 con 0x7fdf50103550 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf56ebe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010e820 con 0x7fdf501087b0 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 0x7fdf50101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:19.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 0x7fdf50101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41188/0 (socket says 172.21.15.38:41188) 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 -- 172.21.15.38:0/1815149446 learned_addr learned my addr 172.21.15.38:0/1815149446 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf5545b700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf501087b0 0x7fdf50101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf54c5a700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf50103550 0x7fdf50102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 -- 172.21.15.38:0/1815149446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf501087b0 msgr2=0x7fdf50101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf501087b0 0x7fdf50101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 -- 172.21.15.38:0/1815149446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf50103550 msgr2=0x7fdf50102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf50103550 0x7fdf50102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 -- 172.21.15.38:0/1815149446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf501022f0 con 0x7fdf50105e80 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4ffff700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 0x7fdf50101500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c9b0 tx=0x7fdf4400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44007b00 con 0x7fdf50105e80 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf501a2d30 con 0x7fdf50105e80 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdf44007ca0 con 0x7fdf50105e80 2024-04-25T13:04:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.440+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44018640 con 0x7fdf50105e80 2024-04-25T13:04:19.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.441+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501a3210 con 0x7fdf50105e80 2024-04-25T13:04:19.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.442+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf50069730 con 0x7fdf50105e80 2024-04-25T13:04:19.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.443+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf440188a0 con 0x7fdf50105e80 2024-04-25T13:04:19.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.443+0000 7fdf4dffb700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf380a3c40 0x7fdf380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:19.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.443+0000 7fdf54c5a700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf380a3c40 0x7fdf380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:19.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.443+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdf440d03c0 con 0x7fdf50105e80 2024-04-25T13:04:19.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.444+0000 7fdf54c5a700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf380a3c40 0x7fdf380a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005950 tx=0x7fdf400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:19.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.448+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf44014030 con 0x7fdf50105e80 2024-04-25T13:04:19.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:19 smithi104 ceph-mon[32726]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:19.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.664+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdf50062e80 con 0x7fdf380a3c40 2024-04-25T13:04:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.678+0000 7fdf4dffb700 1 -- 172.21.15.38:0/1815149446 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fdf50062e80 con 0x7fdf380a3c40 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.a smithi038 running (102s) 17s ago 101s 19.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a7e2e7b9f3ba 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.b smithi104 running (99s) 33s ago 98s 26.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8270cd36762e 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.c smithi178 running (113s) 33s ago 113s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 13ba5e7f6fdc 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.d smithi038 running (2m) 17s ago 2m 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 40c4b9555870 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.e smithi104 running (119s) 33s ago 118s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e1d286e0388c 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.f smithi178 running (116s) 33s ago 116s 17.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 89bffe7e0e36 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.g smithi038 running (105s) 17s ago 104s 16.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3bb9cf690114 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.h smithi104 running (110s) 33s ago 110s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 aa0d6bbbcaf9 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.i smithi178 running (107s) 33s ago 107s 17.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2a632e5a93d5 2024-04-25T13:04:19.677 INFO:teuthology.orchestra.run.smithi038.stdout:mds.j smithi038 running (2m) 17s ago 2m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6fe915df10dd 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mds.k smithi104 running (2m) 33s ago 2m 15.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7ea9de500687 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mds.l smithi178 running (2m) 33s ago 2m 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 498a42164513 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.x smithi038 *:9283,8765 running (9m) 17s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e85391fcd684 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.y smithi104 *:8443,8765 running (7m) 33s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f44d51b5520c 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.z smithi178 *:8443,8765 running (7m) 33s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 07d832782326 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mon.a smithi038 running (9m) 17s ago 9m 53.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 9c0a6e656491 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mon.b smithi104 running (7m) 33s ago 7m 38.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 a92d3a01a46a 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:mon.c smithi178 running (8m) 33s ago 7m 39.2M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 fd042fa8a8aa 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 17s ago 7m 54.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 7a23938e979a 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi104 running (6m) 33s ago 6m 60.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f833e7711f31 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi178 running (6m) 33s ago 6m 45.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 227864c65b63 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi038 running (6m) 17s ago 6m 45.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 95d4bb102f1f 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi104 running (5m) 33s ago 5m 42.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 45cdd57061e1 2024-04-25T13:04:19.678 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi178 running (5m) 33s ago 5m 53.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 28e595671c81 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 smithi038 running (4m) 17s ago 4m 52.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 e28f69ffbbed 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 smithi104 running (4m) 33s ago 4m 55.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 19d2f4d2b815 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 smithi178 running (3m) 33s ago 3m 56.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0e97cc17e8ad 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 smithi038 running (3m) 17s ago 3m 52.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b470e41bdd20 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 smithi104 running (2m) 33s ago 2m 45.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d4b5d15b3124 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 smithi178 running (2m) 33s ago 2m 41.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 34c1b2ca4a3a 2024-04-25T13:04:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf380a3c40 msgr2=0x7fdf380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf380a3c40 0x7fdf380a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005950 tx=0x7fdf400058e0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 msgr2=0x7fdf50101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 0x7fdf50101500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c9b0 tx=0x7fdf4400ccc0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 shutdown_connections 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdf380a3c40 0x7fdf380a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf50103550 0x7fdf50102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf501087b0 0x7fdf50101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 --2- 172.21.15.38:0/1815149446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf50105e80 0x7fdf50101500 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.680+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 >> 172.21.15.38:0/1815149446 conn(0x7fdf500fcf60 msgr2=0x7fdf50103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.681+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 shutdown_connections 2024-04-25T13:04:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:19.681+0000 7fdf56ebe700 1 -- 172.21.15.38:0/1815149446 wait complete. 2024-04-25T13:04:20.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:04:20.635 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:21.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:21 smithi038 ceph-mon[29400]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:21.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:21 smithi038 ceph-mon[29400]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:21.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:21 smithi178 ceph-mon[33233]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:21.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:21 smithi178 ceph-mon[33233]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:21.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:21 smithi104 ceph-mon[32726]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T13:04:21.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:21 smithi104 ceph-mon[32726]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.407+0000 7f0466360700 1 -- 172.21.15.38:0/1701180546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460103cc0 msgr2=0x7f0460104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.407+0000 7f0466360700 1 --2- 172.21.15.38:0/1701180546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460103cc0 0x7f0460104120 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04540099c0 tx=0x7f0454009cd0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.407+0000 7f0466360700 1 -- 172.21.15.38:0/1701180546 shutdown_connections 2024-04-25T13:04:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.407+0000 7f0466360700 1 --2- 172.21.15.38:0/1701180546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0460104660 0x7f046010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.407+0000 7f0466360700 1 --2- 172.21.15.38:0/1701180546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0460102ac0 0x7f0460102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.407+0000 7f0466360700 1 --2- 172.21.15.38:0/1701180546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460103cc0 0x7f0460104120 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.408+0000 7f0466360700 1 -- 172.21.15.38:0/1701180546 >> 172.21.15.38:0/1701180546 conn(0x7f04600fe060 msgr2=0x7f04601004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.408+0000 7f0466360700 1 -- 172.21.15.38:0/1701180546 shutdown_connections 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.408+0000 7f0466360700 1 -- 172.21.15.38:0/1701180546 wait complete. 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.408+0000 7f0466360700 1 Processor -- start 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.408+0000 7f0466360700 1 -- start start 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f0466360700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0460102ac0 0x7f0460198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f0466360700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0460103cc0 0x7f0460199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f0466360700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 0x7f04601a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f0466360700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f046010e790 con 0x7f0460104660 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f0466360700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f046010e610 con 0x7f0460102ac0 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f0466360700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f046010e910 con 0x7f0460103cc0 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 0x7f04601a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 0x7f04601a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40198/0 (socket says 172.21.15.38:40198) 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 -- 172.21.15.38:0/2230420783 learned_addr learned my addr 172.21.15.38:0/2230420783 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f045f7fe700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0460103cc0 0x7f0460199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f045ffff700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0460102ac0 0x7f0460198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 -- 172.21.15.38:0/2230420783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0460103cc0 msgr2=0x7f0460199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0460103cc0 0x7f0460199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 -- 172.21.15.38:0/2230420783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0460102ac0 msgr2=0x7f0460198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0460102ac0 0x7f0460198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 -- 172.21.15.38:0/2230420783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04601a0a20 con 0x7f0460104660 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f04648fd700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 0x7f04601a0380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f045800b9a0 tx=0x7f045800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.409+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800d850 con 0x7f0460104660 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.410+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601a0d10 con 0x7f0460104660 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.410+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f045800e420 con 0x7f0460104660 2024-04-25T13:04:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.410+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458017660 con 0x7f0460104660 2024-04-25T13:04:22.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.410+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04601a1220 con 0x7f0460104660 2024-04-25T13:04:22.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.411+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0460069730 con 0x7f0460104660 2024-04-25T13:04:22.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.412+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f045800e5d0 con 0x7f0460104660 2024-04-25T13:04:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.413+0000 7f045d7fa700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04480a3c40 0x7f04480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.413+0000 7f045ffff700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04480a3c40 0x7f04480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.413+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f04580cffa0 con 0x7f0460104660 2024-04-25T13:04:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.413+0000 7f045ffff700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04480a3c40 0x7f04480a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0460103b20 tx=0x7f0450009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.418+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0458013030 con 0x7f0460104660 2024-04-25T13:04:22.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.634+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0460062e80 con 0x7f04480a3c40 2024-04-25T13:04:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.640+0000 7f045d7fa700 1 -- 172.21.15.38:0/2230420783 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0460062e80 con 0x7f04480a3c40 2024-04-25T13:04:22.638 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:04:22.638 INFO:teuthology.orchestra.run.smithi038.stdout:mds.all 12/12 36s ago 2m smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-04-25T13:04:22.639 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 3/3 36s ago 7m smithi038=x;smithi104=y;smithi178=z;count:3 2024-04-25T13:04:22.639 INFO:teuthology.orchestra.run.smithi038.stdout:mon 3/3 36s ago 8m smithi038:172.21.15.38=a;smithi104:172.21.15.104=b;smithi178:172.21.15.178=c;count:3 2024-04-25T13:04:22.639 INFO:teuthology.orchestra.run.smithi038.stdout:osd 12 36s ago - 2024-04-25T13:04:22.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04480a3c40 msgr2=0x7f04480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04480a3c40 0x7f04480a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0460103b20 tx=0x7f0450009190 comp rx=0 tx=0).stop 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 msgr2=0x7f04601a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 0x7f04601a0380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f045800b9a0 tx=0x7f045800bcb0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 shutdown_connections 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f04480a3c40 0x7f04480a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0460102ac0 0x7f0460198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460104660 0x7f04601a0380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 --2- 172.21.15.38:0/2230420783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0460103cc0 0x7f0460199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 >> 172.21.15.38:0/2230420783 conn(0x7f04600fe060 msgr2=0x7f0460106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 shutdown_connections 2024-04-25T13:04:22.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:22.642+0000 7f0466360700 1 -- 172.21.15.38:0/2230420783 wait complete. 2024-04-25T13:04:23.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:23 smithi038 ceph-mon[29400]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:23.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:04:23.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:23 smithi178 ceph-mon[33233]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:23.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:23 smithi104 ceph-mon[32726]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:04:23.671 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:24.486 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:24 smithi038 ceph-mon[29400]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:24.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:24 smithi178 ceph-mon[33233]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:24.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:24 smithi104 ceph-mon[32726]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 -- 172.21.15.38:0/1330823745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440103cc0 msgr2=0x7f7440104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1330823745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440103cc0 0x7f7440104120 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f742c0099c0 tx=0x7f742c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 -- 172.21.15.38:0/1330823745 shutdown_connections 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1330823745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440102ac0 0x7f7440102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1330823745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440104660 0x7f744010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1330823745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440103cc0 0x7f7440104120 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 -- 172.21.15.38:0/1330823745 >> 172.21.15.38:0/1330823745 conn(0x7f74400fe060 msgr2=0x7f74401004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 -- 172.21.15.38:0/1330823745 shutdown_connections 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.341+0000 7f7446f4a700 1 -- 172.21.15.38:0/1330823745 wait complete. 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 Processor -- start 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 -- start start 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 0x7f7440198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440103cc0 0x7f74401991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440104660 0x7f74401a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f744010e670 con 0x7f7440102ac0 2024-04-25T13:04:25.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f744010e4f0 con 0x7f7440103cc0 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7446f4a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f744010e7f0 con 0x7f7440104660 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 0x7f7440198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 0x7f7440198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40214/0 (socket says 172.21.15.38:40214) 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 -- 172.21.15.38:0/1868139877 learned_addr learned my addr 172.21.15.38:0/1868139877 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f74454e7700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440104660 0x7f74401a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f743ffff700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440103cc0 0x7f74401991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 -- 172.21.15.38:0/1868139877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440104660 msgr2=0x7f74401a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440104660 0x7f74401a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 -- 172.21.15.38:0/1868139877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440103cc0 msgr2=0x7f74401991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440103cc0 0x7f74401991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 -- 172.21.15.38:0/1868139877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74401a08e0 con 0x7f7440102ac0 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f743ffff700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440103cc0 0x7f74401991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.342+0000 7f7444ce6700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 0x7f7440198c60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f743400c990 tx=0x7f743400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.343+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434007b00 con 0x7f7440102ac0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.343+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7434007ca0 con 0x7f7440102ac0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.343+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74401a0bd0 con 0x7f7440102ac0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.343+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434018660 con 0x7f7440102ac0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.343+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74401a1160 con 0x7f7440102ac0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.344+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7440069730 con 0x7f7440102ac0 2024-04-25T13:04:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.345+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7434018800 con 0x7f7440102ac0 2024-04-25T13:04:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.345+0000 7f743dffb700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f74300a3cc0 0x7f74300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.346+0000 7f743ffff700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f74300a3cc0 0x7f74300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.346+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f74340d0360 con 0x7f7440102ac0 2024-04-25T13:04:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.346+0000 7f743ffff700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f74300a3cc0 0x7f74300a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f744019a170 tx=0x7f742c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.350+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7434014030 con 0x7f7440102ac0 2024-04-25T13:04:25.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:25 smithi178 ceph-mon[33233]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:25.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.563+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7440062e80 con 0x7f74300a3cc0 2024-04-25T13:04:25.562 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:25 smithi038 ceph-mon[29400]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:25.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.565+0000 7f743dffb700 1 -- 172.21.15.38:0/1868139877 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7440062e80 con 0x7f74300a3cc0 2024-04-25T13:04:25.563 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:04:25.563 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-04-25T13:04:25.563 INFO:teuthology.orchestra.run.smithi038.stdout:smithi104 172.21.15.104 2024-04-25T13:04:25.563 INFO:teuthology.orchestra.run.smithi038.stdout:smithi178 172.21.15.178 2024-04-25T13:04:25.563 INFO:teuthology.orchestra.run.smithi038.stdout:3 hosts in cluster 2024-04-25T13:04:25.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f74300a3cc0 msgr2=0x7f74300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:25.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f74300a3cc0 0x7f74300a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f744019a170 tx=0x7f742c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 msgr2=0x7f7440198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:25.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 0x7f7440198c60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f743400c990 tx=0x7f743400cca0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 shutdown_connections 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f74300a3cc0 0x7f74300a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440104660 0x7f74401a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7440102ac0 0x7f7440198c60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 --2- 172.21.15.38:0/1868139877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7440103cc0 0x7f74401991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 >> 172.21.15.38:0/1868139877 conn(0x7f74400fe060 msgr2=0x7f7440106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 shutdown_connections 2024-04-25T13:04:25.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:25.567+0000 7f7446f4a700 1 -- 172.21.15.38:0/1868139877 wait complete. 2024-04-25T13:04:25.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:25 smithi104 ceph-mon[32726]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:26.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:04:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:27.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:27 smithi038 ceph-mon[29400]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:27.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:27 smithi038 ceph-mon[29400]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:27.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:27 smithi178 ceph-mon[33233]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:27.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:27 smithi178 ceph-mon[33233]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:27.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:27 smithi104 ceph-mon[32726]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:27.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:27 smithi104 ceph-mon[32726]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 -- 172.21.15.38:0/1077996818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f384810f080 msgr2=0x7f3848111470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 --2- 172.21.15.38:0/1077996818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f384810f080 0x7f3848111470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f384000b0a0 tx=0x7f384000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 -- 172.21.15.38:0/1077996818 shutdown_connections 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 --2- 172.21.15.38:0/1077996818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3848076a70 0x7f3848074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 --2- 172.21.15.38:0/1077996818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f384810f080 0x7f3848111470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 --2- 172.21.15.38:0/1077996818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38480754a0 0x7f3848075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 -- 172.21.15.38:0/1077996818 >> 172.21.15.38:0/1077996818 conn(0x7f38480fe040 msgr2=0x7f3848100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 -- 172.21.15.38:0/1077996818 shutdown_connections 2024-04-25T13:04:28.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.344+0000 7f384ccf3700 1 -- 172.21.15.38:0/1077996818 wait complete. 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 Processor -- start 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 -- start start 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480754a0 0x7f384819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 0x7f384819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f384810f080 0x7f38481a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3848114cb0 con 0x7f3848076a70 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3848114b30 con 0x7f38480754a0 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384ccf3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3848114e30 con 0x7f384810f080 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 0x7f384819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 0x7f384819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40228/0 (socket says 172.21.15.38:40228) 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 -- 172.21.15.38:0/3652597684 learned_addr learned my addr 172.21.15.38:0/3652597684 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f384659c700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480754a0 0x7f384819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3846d9d700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f384810f080 0x7f38481a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 -- 172.21.15.38:0/3652597684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f384810f080 msgr2=0x7f38481a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f384810f080 0x7f38481a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 -- 172.21.15.38:0/3652597684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480754a0 msgr2=0x7f384819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480754a0 0x7f384819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.345+0000 7f3845d9b700 1 -- 172.21.15.38:0/3652597684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38481a6ec0 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.346+0000 7f3845d9b700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 0x7f384819f7f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f383c00d9b0 tx=0x7f383c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.346+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c017030 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.346+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38481a71b0 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.346+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f383c0044f0 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.346+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c003aa0 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.346+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38481a7740 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.347+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3848069730 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.348+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f383c01a070 con 0x7f3848076a70 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.348+0000 7f38377fe700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f38300a3c40 0x7f38300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:28.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.349+0000 7f384659c700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f38300a3c40 0x7f38300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:28.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.349+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f383c0cfd90 con 0x7f3848076a70 2024-04-25T13:04:28.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.349+0000 7f384659c700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f38300a3c40 0x7f38300a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3838005950 tx=0x7f383800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:28.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.353+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f383c09c730 con 0x7f3848076a70 2024-04-25T13:04:28.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.568+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3848062e80 con 0x7f38300a3c40 2024-04-25T13:04:28.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.570+0000 7f38377fe700 1 -- 172.21.15.38:0/3652597684 <== mgr.14152 v2:172.21.15.38:6800/1333793113 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3848062e80 con 0x7f38300a3c40 2024-04-25T13:04:28.568 INFO:teuthology.orchestra.run.smithi038.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:04:28.569 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:04:28.569 INFO:teuthology.orchestra.run.smithi038.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:04:28.569 INFO:teuthology.orchestra.run.smithi038.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f38300a3c40 msgr2=0x7f38300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f38300a3c40 0x7f38300a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3838005950 tx=0x7f383800b410 comp rx=0 tx=0).stop 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 msgr2=0x7f384819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 0x7f384819f7f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f383c00d9b0 tx=0x7f383c00dcc0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 shutdown_connections 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f38300a3c40 0x7f38300a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480754a0 0x7f384819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f384810f080 0x7f38481a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 --2- 172.21.15.38:0/3652597684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3848076a70 0x7f384819f7f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:28.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.572+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 >> 172.21.15.38:0/3652597684 conn(0x7f38480fe040 msgr2=0x7f3848102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:28.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.573+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 shutdown_connections 2024-04-25T13:04:28.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:28.573+0000 7f384ccf3700 1 -- 172.21.15.38:0/3652597684 wait complete. 2024-04-25T13:04:29.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:29 smithi038 ceph-mon[29400]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:29.417 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:04:29.426 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:29.426 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:04:29.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:29 smithi178 ceph-mon[33233]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:29.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:29 smithi104 ceph-mon[32726]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:30.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:30 smithi038 ceph-mon[29400]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:30.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:30 smithi178 ceph-mon[33233]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:30.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:30 smithi104 ceph-mon[32726]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:31.267 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:31 smithi038 ceph-mon[29400]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:31.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.489+0000 7f62ad180700 1 -- 172.21.15.38:0/1509143624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8101220 msgr2=0x7f62a8101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.489+0000 7f62ad180700 1 --2- 172.21.15.38:0/1509143624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8101220 0x7f62a8101640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f62980099c0 tx=0x7f6298009cd0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 -- 172.21.15.38:0/1509143624 shutdown_connections 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 --2- 172.21.15.38:0/1509143624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a8102cb0 0x7f62a8109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 --2- 172.21.15.38:0/1509143624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8101220 0x7f62a8101640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 --2- 172.21.15.38:0/1509143624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8102310 0x7f62a8102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 -- 172.21.15.38:0/1509143624 >> 172.21.15.38:0/1509143624 conn(0x7f62a80fc290 msgr2=0x7f62a80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 -- 172.21.15.38:0/1509143624 shutdown_connections 2024-04-25T13:04:31.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 -- 172.21.15.38:0/1509143624 wait complete. 2024-04-25T13:04:31.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:31 smithi178 ceph-mon[33233]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:31.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.490+0000 7f62ad180700 1 Processor -- start 2024-04-25T13:04:31.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 -- start start 2024-04-25T13:04:31.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a8101220 0x7f62a8105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:31.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 0x7f62a81036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8102cb0 0x7f62a8103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f62a810cd00 con 0x7f62a8102310 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62a810cb80 con 0x7f62a8102cb0 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62ad180700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62a810ce80 con 0x7f62a8101220 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 0x7f62a81036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 0x7f62a81036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47304/0 (socket says 172.21.15.38:47304) 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 -- 172.21.15.38:0/2017435340 learned_addr learned my addr 172.21.15.38:0/2017435340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a6d9d700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a8101220 0x7f62a8105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 -- 172.21.15.38:0/2017435340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a8101220 msgr2=0x7f62a8105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a8101220 0x7f62a8105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 -- 172.21.15.38:0/2017435340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8102cb0 msgr2=0x7f62a8103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8102cb0 0x7f62a8103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 -- 172.21.15.38:0/2017435340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a8104500 con 0x7f62a8102310 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.491+0000 7f62a659c700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 0x7f62a81036f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c9b0 tx=0x7f629c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:31.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.492+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c007b00 con 0x7f62a8102310 2024-04-25T13:04:31.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.492+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a81a2cf0 con 0x7f62a8102310 2024-04-25T13:04:31.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.492+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f629c007ca0 con 0x7f62a8102310 2024-04-25T13:04:31.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.492+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c018640 con 0x7f62a8102310 2024-04-25T13:04:31.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.492+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a31a0 con 0x7f62a8102310 2024-04-25T13:04:31.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.493+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a802df80 con 0x7f62a8102310 2024-04-25T13:04:31.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.494+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f629c0188a0 con 0x7f62a8102310 2024-04-25T13:04:31.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.494+0000 7f628ffff700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f62900a3c40 0x7f62900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:31.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.494+0000 7f62a6d9d700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f62900a3c40 0x7f62900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:31.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.495+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f629c0d0450 con 0x7f62a8102310 2024-04-25T13:04:31.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.495+0000 7f62a6d9d700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f62900a3c40 0x7f62900a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6298006010 tx=0x7f6298009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:31.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.499+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629c014030 con 0x7f62a8102310 2024-04-25T13:04:31.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:31 smithi104 ceph-mon[32726]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.752+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f62a8063650 con 0x7f62a8102310 2024-04-25T13:04:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.752+0000 7f628ffff700 1 -- 172.21.15.38:0/2017435340 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3094 (secure 0 0 0) 0x7f629c09ce30 con 0x7f62a8102310 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:e26 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 26 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:04:31.751 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-04-25T13:02:44.261324+0000 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-04-25T13:02:51.699933+0000 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 300 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 5 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1,2,3,4 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24518,1=24484,2=24482,3=24466,4=14625} 2024-04-25T13:04:31.752 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 0 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 24518 members: 24484,24518,24482,24466,14625 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b{0:24518} state up:active seq 3 addr [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.f{0:24454} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/2362992725,v1:172.21.15.178:6835/2362992725] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.i{1:24484} state up:active seq 5 addr [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.e{1:24443} state up:standby-replay seq 1 addr [v2:172.21.15.104:6834/4172911078,v1:172.21.15.104:6835/4172911078] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.753 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.h{2:24482} state up:active seq 6 addr [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.k{2:24416} state up:standby-replay seq 1 addr [v2:172.21.15.104:6832/650369302,v1:172.21.15.104:6833/650369302] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c{3:24466} state up:active seq 7 addr [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.l{3:24415} state up:standby-replay seq 1 addr [v2:172.21.15.178:6832/2494647672,v1:172.21.15.178:6833/2494647672] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a{4:14625} state up:active seq 5 addr [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.38:6838/3427364907,v1:172.21.15.38:6839/3427364907] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.j{-1:14529} state up:standby seq 1 addr [v2:172.21.15.38:6834/3785884353,v1:172.21.15.38:6835/3785884353] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.754 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d{-1:14541} state up:standby seq 1 addr [v2:172.21.15.38:6836/1490443710,v1:172.21.15.38:6837/1490443710] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:04:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f62900a3c40 msgr2=0x7f62900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f62900a3c40 0x7f62900a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6298006010 tx=0x7f6298009f10 comp rx=0 tx=0).stop 2024-04-25T13:04:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 msgr2=0x7f62a81036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 0x7f62a81036f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c9b0 tx=0x7f629c00ccc0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 shutdown_connections 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f62900a3c40 0x7f62900a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8102cb0 0x7f62a8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a8101220 0x7f62a8105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 --2- 172.21.15.38:0/2017435340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62a8102310 0x7f62a81036f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 >> 172.21.15.38:0/2017435340 conn(0x7f62a80fc290 msgr2=0x7f62a80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 shutdown_connections 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:31.754+0000 7f62ad180700 1 -- 172.21.15.38:0/2017435340 wait complete. 2024-04-25T13:04:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:04:32.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2017435340' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:04:32.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2017435340' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:04:32.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2017435340' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:04:32.609 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:04:32.866 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:33.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:33 smithi038 ceph-mon[29400]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:04:33.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:33 smithi178 ceph-mon[33233]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:04:33.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:33 smithi104 ceph-mon[32726]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T13:04:34.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 -- 172.21.15.38:0/3808123359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8103550 msgr2=0x7f94e8105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3808123359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8103550 0x7f94e8105940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009990 tx=0x7f94d8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 -- 172.21.15.38:0/3808123359 shutdown_connections 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3808123359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e81087b0 0x7f94e810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3808123359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8103550 0x7f94e8105940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3808123359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e8105e80 0x7f94e8108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 -- 172.21.15.38:0/3808123359 >> 172.21.15.38:0/3808123359 conn(0x7f94e80fcf60 msgr2=0x7f94e80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 -- 172.21.15.38:0/3808123359 shutdown_connections 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.689+0000 7f94ed67e700 1 -- 172.21.15.38:0/3808123359 wait complete. 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.690+0000 7f94ed67e700 1 Processor -- start 2024-04-25T13:04:34.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.690+0000 7f94ed67e700 1 -- start start 2024-04-25T13:04:34.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.690+0000 7f94ed67e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8103550 0x7f94e8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:34.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.690+0000 7f94ed67e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 0x7f94e8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.690+0000 7f94ed67e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e81087b0 0x7f94e81a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.690+0000 7f94ed67e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810e680 con 0x7f94e8105e80 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94ed67e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810e500 con 0x7f94e81087b0 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94ed67e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810e800 con 0x7f94e8103550 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 0x7f94e8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 0x7f94e8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47318/0 (socket says 172.21.15.38:47318) 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 -- 172.21.15.38:0/3368034585 learned_addr learned my addr 172.21.15.38:0/3368034585 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e77fe700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e81087b0 0x7f94e81a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e6ffd700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8103550 0x7f94e8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 -- 172.21.15.38:0/3368034585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8103550 msgr2=0x7f94e8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8103550 0x7f94e8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 -- 172.21.15.38:0/3368034585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e81087b0 msgr2=0x7f94e81a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:34.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e81087b0 0x7f94e81a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 -- 172.21.15.38:0/3368034585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e81a08a0 con 0x7f94e8105e80 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e6ffd700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8103550 0x7f94e8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94e67fc700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 0x7f94e8199190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f94d000c920 tx=0x7f94d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0007b00 con 0x7f94e8105e80 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94d0004d00 con 0x7f94e8105e80 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.691+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e81a0b90 con 0x7f94e8105e80 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.692+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0005650 con 0x7f94e8105e80 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.692+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81a10d0 con 0x7f94e8105e80 2024-04-25T13:04:34.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.693+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e8069730 con 0x7f94e8105e80 2024-04-25T13:04:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.694+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f94d0007ca0 con 0x7f94e8105e80 2024-04-25T13:04:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.694+0000 7f94dffff700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94d40a3c40 0x7f94d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.695+0000 7f94e6ffd700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94d40a3c40 0x7f94d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.695+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f94d00d0350 con 0x7f94e8105e80 2024-04-25T13:04:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.695+0000 7f94e6ffd700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94d40a3c40 0x7f94d40a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94d8003910 tx=0x7f94d800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:34.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.698+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94d0014030 con 0x7f94e8105e80 2024-04-25T13:04:34.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.907+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f94e802e000 con 0x7f94e8105e80 2024-04-25T13:04:34.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.908+0000 7f94dffff700 1 -- 172.21.15.38:0/3368034585 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5292 (secure 0 0 0) 0x7f94d0004eb0 con 0x7f94e8105e80 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 88 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:fsid d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-04-25T12:54:40.451583+0000 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-04-25T13:03:37.115660+0000 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:crush_version 26 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:full_ratio 0.95 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:backfillfull_ratio 0.9 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:nearfull_ratio 0.85 2024-04-25T13:04:34.907 INFO:teuthology.orchestra.run.smithi038.stdout:require_min_compat_client luminous 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:min_compat_client luminous 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:require_osd_release squid 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:stretch_mode_enabled false 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:max_osd 12 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6802/4061500171,v1:172.21.15.38:6803/4061500171] [v2:172.21.15.38:6804/4061500171,v1:172.21.15.38:6805/4061500171] exists,up fc0ea8fe-71f4-4976-8844-d906980de942 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/739328309,v1:172.21.15.104:6801/739328309] [v2:172.21.15.104:6802/739328309,v1:172.21.15.104:6803/739328309] exists,up 61eee122-04d2-413a-9203-29184e87e08f 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/760031576,v1:172.21.15.178:6801/760031576] [v2:172.21.15.178:6802/760031576,v1:172.21.15.178:6803/760031576] exists,up d0d8bc3e-b869-46b9-b674-3f547e26ae93 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6810/1943053505,v1:172.21.15.38:6811/1943053505] [v2:172.21.15.38:6812/1943053505,v1:172.21.15.38:6813/1943053505] exists,up 408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/309827173,v1:172.21.15.104:6809/309827173] [v2:172.21.15.104:6810/309827173,v1:172.21.15.104:6811/309827173] exists,up e42ba42a-a706-42ba-84d0-85ef48660c60 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/1085104409,v1:172.21.15.178:6809/1085104409] [v2:172.21.15.178:6810/1085104409,v1:172.21.15.178:6811/1085104409] exists,up 93a59196-6a9a-4233-ad44-dee831b56089 2024-04-25T13:04:34.908 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6818/2531974702,v1:172.21.15.38:6819/2531974702] [v2:172.21.15.38:6820/2531974702,v1:172.21.15.38:6821/2531974702] exists,up fcf8f123-b8f9-49e5-bca5-34f32285c0a5 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/3955477,v1:172.21.15.104:6817/3955477] [v2:172.21.15.104:6818/3955477,v1:172.21.15.104:6819/3955477] exists,up 2f074366-1e63-4ad6-9f21-30018b0cb493 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/2770931166,v1:172.21.15.178:6817/2770931166] [v2:172.21.15.178:6818/2770931166,v1:172.21.15.178:6819/2770931166] exists,up 09d79a36-ce86-4e9a-b60a-6a709d437bc4 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6826/1397570004,v1:172.21.15.38:6827/1397570004] [v2:172.21.15.38:6828/1397570004,v1:172.21.15.38:6829/1397570004] exists,up 47dbcc3a-1a6a-4896-a5d2-9866e080fd7b 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/2081501725,v1:172.21.15.104:6825/2081501725] [v2:172.21.15.104:6826/2081501725,v1:172.21.15.104:6827/2081501725] exists,up 38c90875-61d4-402f-bee4-af2adb22e185 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/700919223,v1:172.21.15.178:6825/700919223] [v2:172.21.15.178:6826/700919223,v1:172.21.15.178:6827/700919223] exists,up f03efb23-83a5-460c-90be-6aef89f88698 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.b [2,4] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.13 [2,11] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.0 [5,9] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.c [3,9] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.e [5,9] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.21 [4,6] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T13:04:34.909 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.27 [4,8] 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/239401667 expires 2024-04-26T12:55:35.192028+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/3157707598 expires 2024-04-26T12:55:35.192028+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/3913530483 expires 2024-04-26T12:55:08.110358+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2782801986 expires 2024-04-26T12:55:08.110358+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/2310511598 expires 2024-04-26T12:55:08.110358+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/2310511598 expires 2024-04-26T12:55:08.110358+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/1148567137 expires 2024-04-26T12:55:35.192028+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/3957422013 expires 2024-04-26T12:55:08.110358+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/3157707598 expires 2024-04-26T12:55:35.192028+0000 2024-04-25T13:04:34.910 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2195408134 expires 2024-04-26T12:55:35.192028+0000 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94d40a3c40 msgr2=0x7f94d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94d40a3c40 0x7f94d40a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94d8003910 tx=0x7f94d800b540 comp rx=0 tx=0).stop 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 msgr2=0x7f94e8199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 0x7f94e8199190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f94d000c920 tx=0x7f94d000cc30 comp rx=0 tx=0).stop 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 shutdown_connections 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f94d40a3c40 0x7f94d40a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94e81087b0 0x7f94e81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8103550 0x7f94e8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 --2- 172.21.15.38:0/3368034585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e8105e80 0x7f94e8199190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:34.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.910+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 >> 172.21.15.38:0/3368034585 conn(0x7f94e80fcf60 msgr2=0x7f94e8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:34.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.911+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 shutdown_connections 2024-04-25T13:04:34.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:34.911+0000 7f94ed67e700 1 -- 172.21.15.38:0/3368034585 wait complete. 2024-04-25T13:04:35.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:35 smithi178 ceph-mon[33233]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:35.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:35 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3368034585' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:04:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:35 smithi038 ceph-mon[29400]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:35.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:35 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3368034585' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:04:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:35 smithi104 ceph-mon[32726]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:35.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:35 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3368034585' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:04:35.680 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:04:35.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:04:35.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 -- 172.21.15.38:0/2677078392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8107c90 msgr2=0x7f93e810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 --2- 172.21.15.38:0/2677078392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8107c90 0x7f93e810a0e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009ca0 tx=0x7f93e002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 -- 172.21.15.38:0/2677078392 shutdown_connections 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 --2- 172.21.15.38:0/2677078392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8107c90 0x7f93e810a0e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 --2- 172.21.15.38:0/2677078392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93e8105360 0x7f93e8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 --2- 172.21.15.38:0/2677078392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93e8102a30 0x7f93e8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 -- 172.21.15.38:0/2677078392 >> 172.21.15.38:0/2677078392 conn(0x7f93e80fc440 msgr2=0x7f93e80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 -- 172.21.15.38:0/2677078392 shutdown_connections 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.798+0000 7f93ed320640 1 -- 172.21.15.38:0/2677078392 wait complete. 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 Processor -- start 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 -- start start 2024-04-25T13:04:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 0x7f93e81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93e8105360 0x7f93e81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93e8107c90 0x7f93e819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810dc30 con 0x7f93e8102a30 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810dab0 con 0x7f93e8107c90 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93ed320640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810ddb0 con 0x7f93e8105360 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 0x7f93e81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 0x7f93e81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47332/0 (socket says 172.21.15.38:47332) 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93e6ffd640 1 -- 172.21.15.38:0/2899203762 learned_addr learned my addr 172.21.15.38:0/2899203762 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93e77fe640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93e8107c90 0x7f93e819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.799+0000 7f93e67fc640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93e8105360 0x7f93e81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93e6ffd640 1 -- 172.21.15.38:0/2899203762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93e8105360 msgr2=0x7f93e81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93e6ffd640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93e8105360 0x7f93e81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93e6ffd640 1 -- 172.21.15.38:0/2899203762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93e8107c90 msgr2=0x7f93e819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93e6ffd640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93e8107c90 0x7f93e819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93e6ffd640 1 -- 172.21.15.38:0/2899203762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e819fec0 con 0x7f93e8102a30 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93e6ffd640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 0x7f93e81982a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d950 tx=0x7f93d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017030 con 0x7f93e8102a30 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f93d8004020 con 0x7f93e8102a30 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81a01b0 con 0x7f93e8102a30 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80047a0 con 0x7f93e8102a30 2024-04-25T13:04:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.800+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a0700 con 0x7f93e8102a30 2024-04-25T13:04:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.801+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e80fdf30 con 0x7f93e8102a30 2024-04-25T13:04:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.802+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93d80041d0 con 0x7f93e8102a30 2024-04-25T13:04:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.803+0000 7f93cffff640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93d00a3bb0 0x7f93d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.803+0000 7f93e67fc640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93d00a3bb0 0x7f93d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.803+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f93d80d0490 con 0x7f93e8102a30 2024-04-25T13:04:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.803+0000 7f93e67fc640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93d00a3bb0 0x7f93d00a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93e81997b0 tx=0x7f93dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.806+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93d809ce30 con 0x7f93e8102a30 2024-04-25T13:04:35.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.997+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93e8062b50 con 0x7f93e8102a30 2024-04-25T13:04:35.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:35.998+0000 7f93cffff640 1 -- 172.21.15.38:0/2899203762 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22650 (secure 0 0 0) 0x7f93d809c730 con 0x7f93e8102a30 2024-04-25T13:04:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:35.997 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:03:37.115660+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:04:35.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93d00a3bb0 msgr2=0x7f93d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:35.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93d00a3bb0 0x7f93d00a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93e81997b0 tx=0x7f93dc00b390 comp rx=0 tx=0).stop 2024-04-25T13:04:35.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 msgr2=0x7f93e81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:35.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 0x7f93e81982a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d950 tx=0x7f93d800de10 comp rx=0 tx=0).stop 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 shutdown_connections 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f93d00a3bb0 0x7f93d00a6070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93e8107c90 0x7f93e819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93e8105360 0x7f93e81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 --2- 172.21.15.38:0/2899203762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93e8102a30 0x7f93e81982a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.000+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 >> 172.21.15.38:0/2899203762 conn(0x7f93e80fc440 msgr2=0x7f93e8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.001+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 shutdown_connections 2024-04-25T13:04:36.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.001+0000 7f93ed320640 1 -- 172.21.15.38:0/2899203762 wait complete. 2024-04-25T13:04:36.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:02:42.235731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:02:43.206179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:04:36.011 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:04:36.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 -- 172.21.15.38:0/1805599671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04107c90 msgr2=0x7fae0410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 --2- 172.21.15.38:0/1805599671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04107c90 0x7fae0410a0e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009ca0 tx=0x7fadf402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 -- 172.21.15.38:0/1805599671 shutdown_connections 2024-04-25T13:04:36.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 --2- 172.21.15.38:0/1805599671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04107c90 0x7fae0410a0e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 --2- 172.21.15.38:0/1805599671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae04105360 0x7fae04107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 --2- 172.21.15.38:0/1805599671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae04102a30 0x7fae04104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 -- 172.21.15.38:0/1805599671 >> 172.21.15.38:0/1805599671 conn(0x7fae040fc440 msgr2=0x7fae040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 -- 172.21.15.38:0/1805599671 shutdown_connections 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.118+0000 7fae08baf640 1 -- 172.21.15.38:0/1805599671 wait complete. 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 Processor -- start 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 -- start start 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 0x7fae041982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae04105360 0x7fae04198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae04107c90 0x7fae0419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fae0410dc90 con 0x7fae04102a30 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae0410db10 con 0x7fae04105360 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae08baf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae0410de10 con 0x7fae04107c90 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 0x7fae041982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 0x7fae041982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47344/0 (socket says 172.21.15.38:47344) 2024-04-25T13:04:36.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.119+0000 7fae0259c640 1 -- 172.21.15.38:0/1347694300 learned_addr learned my addr 172.21.15.38:0/1347694300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae01d9b640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae04105360 0x7fae04198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae02d9d640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae04107c90 0x7fae0419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae0259c640 1 -- 172.21.15.38:0/1347694300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae04107c90 msgr2=0x7fae0419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae0259c640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae04107c90 0x7fae0419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae0259c640 1 -- 172.21.15.38:0/1347694300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae04105360 msgr2=0x7fae04198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae0259c640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae04105360 0x7fae04198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae0259c640 1 -- 172.21.15.38:0/1347694300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0419fef0 con 0x7fae04102a30 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae0259c640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 0x7fae041982d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00d950 tx=0x7fadfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc017030 con 0x7fae04102a30 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fadfc004020 con 0x7fae04102a30 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae041a01e0 con 0x7fae04102a30 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0047a0 con 0x7fae04102a30 2024-04-25T13:04:36.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.120+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae041a0780 con 0x7fae04102a30 2024-04-25T13:04:36.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.121+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae040fdf30 con 0x7fae04102a30 2024-04-25T13:04:36.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.122+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fadfc0041d0 con 0x7fae04102a30 2024-04-25T13:04:36.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.123+0000 7fadfb7fe640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fadf00a3c30 0x7fadf00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.123+0000 7fae01d9b640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fadf00a3c30 0x7fadf00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.123+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fadfc0cfe90 con 0x7fae04102a30 2024-04-25T13:04:36.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.123+0000 7fae01d9b640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fadf00a3c30 0x7fadf00a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae041997e0 tx=0x7fadec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:36.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.127+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadfc09c870 con 0x7fae04102a30 2024-04-25T13:04:36.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:36 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2899203762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:36.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.327+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae04062b50 con 0x7fae04102a30 2024-04-25T13:04:36.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.327+0000 7fadfb7fe640 1 -- 172.21.15.38:0/1347694300 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fadfc09c170 con 0x7fae04102a30 2024-04-25T13:04:36.326 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:04:36.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fadf00a3c30 msgr2=0x7fadf00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fadf00a3c30 0x7fadf00a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae041997e0 tx=0x7fadec008040 comp rx=0 tx=0).stop 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 msgr2=0x7fae041982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 0x7fae041982d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00d950 tx=0x7fadfc00de10 comp rx=0 tx=0).stop 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 shutdown_connections 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fadf00a3c30 0x7fadf00a60f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae04107c90 0x7fae0419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae04105360 0x7fae04198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 --2- 172.21.15.38:0/1347694300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae04102a30 0x7fae041982d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 >> 172.21.15.38:0/1347694300 conn(0x7fae040fc440 msgr2=0x7fae04103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 shutdown_connections 2024-04-25T13:04:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.330+0000 7fae08baf640 1 -- 172.21.15.38:0/1347694300 wait complete. 2024-04-25T13:04:36.339 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1887499559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 msgr2=0x7f9f1c1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1887499559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c1092a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009ca0 tx=0x7f9f1402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1887499559 shutdown_connections 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1887499559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c1092a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1887499559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f1c101fc0 0x7f9f1c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1887499559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 0x7f9f1c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1887499559 >> 172.21.15.38:0/1887499559 conn(0x7f9f1c0fc3e0 msgr2=0x7f9f1c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1887499559 shutdown_connections 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1887499559 wait complete. 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 Processor -- start 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.446+0000 7f9f20d0f640 1 -- start start 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 0x7f9f1c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f1c101fc0 0x7f9f1c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c10dc80 con 0x7f9f1c102980 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c10db00 con 0x7f9f1c100e70 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c10de00 con 0x7f9f1c101fc0 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47360/0 (socket says 172.21.15.38:47360) 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 -- 172.21.15.38:0/1446564635 learned_addr learned my addr 172.21.15.38:0/1446564635 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f13fff640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f1c101fc0 0x7f9f1c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1a59c640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 0x7f9f1c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 -- 172.21.15.38:0/1446564635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f1c101fc0 msgr2=0x7f9f1c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f1c101fc0 0x7f9f1c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 -- 172.21.15.38:0/1446564635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 msgr2=0x7f9f1c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 0x7f9f1c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 -- 172.21.15.38:0/1446564635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c19fee0 con 0x7f9f1c102980 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1a59c640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 0x7f9f1c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f1ad9d640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c19f870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009dc0 tx=0x7f9f1402f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14003cf0 con 0x7f9f1c102980 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c1a0170 con 0x7f9f1c102980 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f14004510 con 0x7f9f1c102980 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1402fdb0 con 0x7f9f1c102980 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.447+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c1a0790 con 0x7f9f1c102980 2024-04-25T13:04:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.448+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c108120 con 0x7f9f1c102980 2024-04-25T13:04:36.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.450+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9f140046c0 con 0x7f9f1c102980 2024-04-25T13:04:36.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.450+0000 7f9f137fe640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f080a3bb0 0x7f9f080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.450+0000 7f9f1a59c640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f080a3bb0 0x7f9f080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.450+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9f140f1ca0 con 0x7f9f1c102980 2024-04-25T13:04:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.451+0000 7f9f1a59c640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f080a3bb0 0x7f9f080a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c006fd0 tx=0x7f9f0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.454+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f140be680 con 0x7f9f1c102980 2024-04-25T13:04:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:36 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2899203762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:36 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2899203762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:36.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.641+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f1c062b50 con 0x7f9f1c102980 2024-04-25T13:04:36.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.641+0000 7f9f137fe640 1 -- 172.21.15.38:0/1446564635 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f9f140bdf80 con 0x7f9f1c102980 2024-04-25T13:04:36.640 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T13:04:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f080a3bb0 msgr2=0x7f9f080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f080a3bb0 0x7f9f080a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c006fd0 tx=0x7f9f0c008040 comp rx=0 tx=0).stop 2024-04-25T13:04:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 msgr2=0x7f9f1c19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c19f870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009dc0 tx=0x7f9f1402f460 comp rx=0 tx=0).stop 2024-04-25T13:04:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 shutdown_connections 2024-04-25T13:04:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9f080a3bb0 0x7f9f080a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f1c102980 0x7f9f1c19f870 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f1c101fc0 0x7f9f1c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 --2- 172.21.15.38:0/1446564635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f1c100e70 0x7f9f1c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.643+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 >> 172.21.15.38:0/1446564635 conn(0x7f9f1c0fc3e0 msgr2=0x7f9f1c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.644+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 shutdown_connections 2024-04-25T13:04:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.644+0000 7f9f20d0f640 1 -- 172.21.15.38:0/1446564635 wait complete. 2024-04-25T13:04:36.653 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:04:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 -- 172.21.15.38:0/156427190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330105360 msgr2=0x7fc330107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 --2- 172.21.15.38:0/156427190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330105360 0x7fc330107750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc324009900 tx=0x7fc32402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 -- 172.21.15.38:0/156427190 shutdown_connections 2024-04-25T13:04:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 --2- 172.21.15.38:0/156427190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc330107c90 0x7fc33010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 --2- 172.21.15.38:0/156427190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330105360 0x7fc330107750 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 --2- 172.21.15.38:0/156427190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc330102a30 0x7fc330104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 -- 172.21.15.38:0/156427190 >> 172.21.15.38:0/156427190 conn(0x7fc3300fc440 msgr2=0x7fc3300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 -- 172.21.15.38:0/156427190 shutdown_connections 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.761+0000 7fc335d22640 1 -- 172.21.15.38:0/156427190 wait complete. 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 Processor -- start 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 -- start start 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc330102a30 0x7fc330198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc330105360 0x7fc330198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 0x7fc33019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010dcc0 con 0x7fc330107c90 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010db40 con 0x7fc330102a30 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc335d22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010de40 con 0x7fc330105360 2024-04-25T13:04:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc32ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 0x7fc33019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc32ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 0x7fc33019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47384/0 (socket says 172.21.15.38:47384) 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc32ffff640 1 -- 172.21.15.38:0/3268442348 learned_addr learned my addr 172.21.15.38:0/3268442348 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc32f7fe640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc330102a30 0x7fc330198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc32effd640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc330105360 0x7fc330198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.762+0000 7fc32ffff640 1 -- 172.21.15.38:0/3268442348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc330105360 msgr2=0x7fc330198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32ffff640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc330105360 0x7fc330198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32ffff640 1 -- 172.21.15.38:0/3268442348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc330102a30 msgr2=0x7fc330198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32ffff640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc330102a30 0x7fc330198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32ffff640 1 -- 172.21.15.38:0/3268442348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33019ff60 con 0x7fc330107c90 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32ffff640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 0x7fc33019f8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b970 tx=0x7fc32800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328019070 con 0x7fc330107c90 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3301a0250 con 0x7fc330107c90 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc32800cb40 con 0x7fc330107c90 2024-04-25T13:04:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328007650 con 0x7fc330107c90 2024-04-25T13:04:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.763+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301a0820 con 0x7fc330107c90 2024-04-25T13:04:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.764+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3300fdf30 con 0x7fc330107c90 2024-04-25T13:04:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.765+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc32801c070 con 0x7fc330107c90 2024-04-25T13:04:36.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.765+0000 7fc32cff9640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3180a3c30 0x7fc3180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:36.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.766+0000 7fc32f7fe640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3180a3c30 0x7fc3180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:36.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.766+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc3280cf930 con 0x7fc330107c90 2024-04-25T13:04:36.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.766+0000 7fc32f7fe640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3180a3c30 0x7fc3180a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc32000bbe0 tx=0x7fc32000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:36.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.770+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc32809bcc0 con 0x7fc330107c90 2024-04-25T13:04:36.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.957+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc330062b50 con 0x7fc330107c90 2024-04-25T13:04:36.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.957+0000 7fc32cff9640 1 -- 172.21.15.38:0/3268442348 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc32809ffc0 con 0x7fc330107c90 2024-04-25T13:04:36.956 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T13:04:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3180a3c30 msgr2=0x7fc3180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3180a3c30 0x7fc3180a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc32000bbe0 tx=0x7fc32000b390 comp rx=0 tx=0).stop 2024-04-25T13:04:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 msgr2=0x7fc33019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 0x7fc33019f8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b970 tx=0x7fc32800be30 comp rx=0 tx=0).stop 2024-04-25T13:04:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 shutdown_connections 2024-04-25T13:04:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fc3180a3c30 0x7fc3180a60f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc330107c90 0x7fc33019f8f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc330105360 0x7fc330198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 --2- 172.21.15.38:0/3268442348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc330102a30 0x7fc330198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:36.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.959+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 >> 172.21.15.38:0/3268442348 conn(0x7fc3300fc440 msgr2=0x7fc330103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:36.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.960+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 shutdown_connections 2024-04-25T13:04:36.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:36.960+0000 7fc335d22640 1 -- 172.21.15.38:0/3268442348 wait complete. 2024-04-25T13:04:36.969 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:04:37.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 -- 172.21.15.38:0/3144714993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 msgr2=0x7f62b01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 --2- 172.21.15.38:0/3144714993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b01024c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009900 tx=0x7f62a002a410 comp rx=0 tx=0).stop 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 -- 172.21.15.38:0/3144714993 shutdown_connections 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 --2- 172.21.15.38:0/3144714993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0102a00 0x7f62b0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 --2- 172.21.15.38:0/3144714993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b01024c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 --2- 172.21.15.38:0/3144714993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0100e60 0x7f62b0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 -- 172.21.15.38:0/3144714993 >> 172.21.15.38:0/3144714993 conn(0x7f62b00fc440 msgr2=0x7f62b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 -- 172.21.15.38:0/3144714993 shutdown_connections 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.072+0000 7f62b76e1640 1 -- 172.21.15.38:0/3144714993 wait complete. 2024-04-25T13:04:37.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 Processor -- start 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 -- start start 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0100e60 0x7f62b0197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0102a00 0x7f62b019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f62b010cb30 con 0x7f62b0102040 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62b010c9b0 con 0x7f62b0102a00 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b76e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62b010ccb0 con 0x7f62b0100e60 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b0197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47414/0 (socket says 172.21.15.38:47414) 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 -- 172.21.15.38:0/2069497202 learned_addr learned my addr 172.21.15.38:0/2069497202 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b5c7e640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0102a00 0x7f62b019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b547d640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0100e60 0x7f62b0197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 -- 172.21.15.38:0/2069497202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0100e60 msgr2=0x7f62b0197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0100e60 0x7f62b0197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 -- 172.21.15.38:0/2069497202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0102a00 msgr2=0x7f62b019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0102a00 0x7f62b019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b4c7c640 1 -- 172.21.15.38:0/2069497202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b019ed70 con 0x7f62b0102040 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b547d640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0100e60 0x7f62b0197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.073+0000 7f62b5c7e640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0102a00 0x7f62b019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.074+0000 7f62b4c7c640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b0197690 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62a0005940 tx=0x7f62a00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.074+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a003f030 con 0x7f62b0102040 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.074+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b019f000 con 0x7f62b0102040 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.074+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f62a0003cd0 con 0x7f62b0102040 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.074+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0035070 con 0x7f62b0102040 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.074+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b019f4f0 con 0x7f62b0102040 2024-04-25T13:04:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.075+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b01081a0 con 0x7f62b0102040 2024-04-25T13:04:37.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.076+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62a0042030 con 0x7f62b0102040 2024-04-25T13:04:37.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.076+0000 7f62a67fc640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f629c0a3bb0 0x7f629c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.077+0000 7f62b547d640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f629c0a3bb0 0x7f629c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.077+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f62a00f0d10 con 0x7f62b0102040 2024-04-25T13:04:37.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.077+0000 7f62b547d640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f629c0a3bb0 0x7f629c0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62b0101bb0 tx=0x7f62ac005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:37.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.080+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62a00bd6f0 con 0x7f62b0102040 2024-04-25T13:04:37.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.310+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b002d5c0 con 0x7f62b0102040 2024-04-25T13:04:37.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.311+0000 7f62a67fc640 1 -- 172.21.15.38:0/2069497202 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11938 (secure 0 0 0) 0x7f62a00dd020 con 0x7f62b0102040 2024-04-25T13:04:37.310 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:37.310 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:02:51.699933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24484,24518,24482,24466,14625]},"id":1}]} 2024-04-25T13:04:37.310 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f629c0a3bb0 msgr2=0x7f629c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f629c0a3bb0 0x7f629c0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62b0101bb0 tx=0x7f62ac005e70 comp rx=0 tx=0).stop 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 msgr2=0x7f62b0197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b0197690 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62a0005940 tx=0x7f62a00043b0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 shutdown_connections 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f629c0a3bb0 0x7f629c0a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b0102a00 0x7f62b019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62b0102040 0x7f62b0197690 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 --2- 172.21.15.38:0/2069497202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b0100e60 0x7f62b0197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 >> 172.21.15.38:0/2069497202 conn(0x7f62b00fc440 msgr2=0x7f62b0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 shutdown_connections 2024-04-25T13:04:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.313+0000 7f62b76e1640 1 -- 172.21.15.38:0/2069497202 wait complete. 2024-04-25T13:04:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:37 smithi038 ceph-mon[29400]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:37 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1347694300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:37 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1446564635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:37 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3268442348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.323 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/371362712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74102a20 msgr2=0x7f4e74104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/371362712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74102a20 0x7f4e74104e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009900 tx=0x7f4e7002a410 comp rx=0 tx=0).stop 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/371362712 shutdown_connections 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/371362712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74107c80 0x7f4e7410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/371362712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 0x7f4e74107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/371362712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74102a20 0x7f4e74104e10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/371362712 >> 172.21.15.38:0/371362712 conn(0x7f4e740fc430 msgr2=0x7f4e740fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/371362712 shutdown_connections 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.426+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/371362712 wait complete. 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 Processor -- start 2024-04-25T13:04:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 -- start start 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74102a20 0x7f4e74198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 0x7f4e74198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 0x7f4e7419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7410dc20 con 0x7f4e74107c80 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7410daa0 con 0x7f4e74105350 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7c3c6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7410dda0 con 0x7f4e74102a20 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 0x7f4e7419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 0x7f4e7419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47438/0 (socket says 172.21.15.38:47438) 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 -- 172.21.15.38:0/4024287127 learned_addr learned my addr 172.21.15.38:0/4024287127 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a162640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74102a20 0x7f4e74198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e79961640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 0x7f4e74198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 -- 172.21.15.38:0/4024287127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74102a20 msgr2=0x7f4e74198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74102a20 0x7f4e74198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 -- 172.21.15.38:0/4024287127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 msgr2=0x7f4e74198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 0x7f4e74198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 -- 172.21.15.38:0/4024287127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7419fe40 con 0x7f4e74107c80 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a162640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74102a20 0x7f4e74198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e79961640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 0x7f4e74198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.427+0000 7f4e7a963640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 0x7f4e7419f7d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00eaa0 tx=0x7f4e6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.428+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00cdb0 con 0x7f4e74107c80 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.428+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4e6c005020 con 0x7f4e74107c80 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.428+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e741a0130 con 0x7f4e74107c80 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.428+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0116d0 con 0x7f4e74107c80 2024-04-25T13:04:37.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.428+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e741a06d0 con 0x7f4e74107c80 2024-04-25T13:04:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.429+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e740fdf20 con 0x7f4e74107c80 2024-04-25T13:04:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.430+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e6c0051d0 con 0x7f4e74107c80 2024-04-25T13:04:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.431+0000 7f4e6b7fe640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4e600a3c30 0x7f4e600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.431+0000 7f4e7a162640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4e600a3c30 0x7f4e600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.431+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4e6c0d0620 con 0x7f4e74107c80 2024-04-25T13:04:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.431+0000 7f4e7a162640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4e600a3c30 0x7f4e600a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e70000c00 tx=0x7f4e70031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.435+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e6c016030 con 0x7f4e74107c80 2024-04-25T13:04:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:37 smithi178 ceph-mon[33233]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:37 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1347694300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:37 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1446564635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:37 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3268442348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:37 smithi104 ceph-mon[32726]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T13:04:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:37 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1347694300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:37 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1446564635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:37 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3268442348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:37.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.624+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e74062b50 con 0x7f4e74107c80 2024-04-25T13:04:37.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.626+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/4024287127 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22650 (secure 0 0 0) 0x7f4e6c09d000 con 0x7f4e74107c80 2024-04-25T13:04:37.624 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:37.624 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:03:37.115660+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:04:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.627+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4e600a3c30 msgr2=0x7f4e600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.627+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4e600a3c30 0x7f4e600a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e70000c00 tx=0x7f4e70031510 comp rx=0 tx=0).stop 2024-04-25T13:04:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.627+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 msgr2=0x7f4e7419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.627+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 0x7f4e7419f7d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00eaa0 tx=0x7f4e6c00ef60 comp rx=0 tx=0).stop 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 shutdown_connections 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4e600a3c30 0x7f4e600a60f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e74107c80 0x7f4e7419f7d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e74105350 0x7f4e74198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 --2- 172.21.15.38:0/4024287127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e74102a20 0x7f4e74198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 >> 172.21.15.38:0/4024287127 conn(0x7f4e740fc430 msgr2=0x7f4e741034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 shutdown_connections 2024-04-25T13:04:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.628+0000 7f4e7c3c6640 1 -- 172.21.15.38:0/4024287127 wait complete. 2024-04-25T13:04:37.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:02:42.235731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:02:43.206179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:04:37.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:04:37.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 -- 172.21.15.38:0/1733468665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7681031a0 msgr2=0x7fb768103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 --2- 172.21.15.38:0/1733468665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7681031a0 0x7fb768103620 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009900 tx=0x7fb75c02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 -- 172.21.15.38:0/1733468665 shutdown_connections 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 --2- 172.21.15.38:0/1733468665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb768103b60 0x7fb76810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 --2- 172.21.15.38:0/1733468665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7681031a0 0x7fb768103620 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 --2- 172.21.15.38:0/1733468665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb768101fb0 0x7fb7681023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 -- 172.21.15.38:0/1733468665 >> 172.21.15.38:0/1733468665 conn(0x7fb7680fd520 msgr2=0x7fb7680ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 -- 172.21.15.38:0/1733468665 shutdown_connections 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 -- 172.21.15.38:0/1733468665 wait complete. 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 Processor -- start 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.745+0000 7fb76da05640 1 -- start start 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb768101fb0 0x7fb768075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7681031a0 0x7fb768074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 0x7fb768074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810dc10 con 0x7fb768103b60 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810da90 con 0x7fb768101fb0 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810dd90 con 0x7fb7681031a0 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 0x7fb768074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 0x7fb768074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47446/0 (socket says 172.21.15.38:47446) 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 -- 172.21.15.38:0/2860171675 learned_addr learned my addr 172.21.15.38:0/2860171675 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb766ffd640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb768101fb0 0x7fb768075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7667fc640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7681031a0 0x7fb768074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 -- 172.21.15.38:0/2860171675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7681031a0 msgr2=0x7fb768074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7681031a0 0x7fb768074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 -- 172.21.15.38:0/2860171675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb768101fb0 msgr2=0x7fb768075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb768101fb0 0x7fb768075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 -- 172.21.15.38:0/2860171675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768075110 con 0x7fb768103b60 2024-04-25T13:04:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb7677fe640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 0x7fb768074880 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb760009fd0 tx=0x7fb76000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760017070 con 0x7fb768103b60 2024-04-25T13:04:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb760003e40 con 0x7fb768103b60 2024-04-25T13:04:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7681a2430 con 0x7fb768103b60 2024-04-25T13:04:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7600106c0 con 0x7fb768103b60 2024-04-25T13:04:37.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.746+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7681a2930 con 0x7fb768103b60 2024-04-25T13:04:37.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.747+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb768109340 con 0x7fb768103b60 2024-04-25T13:04:37.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.749+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb760003970 con 0x7fb768103b60 2024-04-25T13:04:37.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.749+0000 7fb76ca03640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:37.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.749+0000 7fb766ffd640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:37.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.749+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb7600d0170 con 0x7fb768103b60 2024-04-25T13:04:37.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.750+0000 7fb766ffd640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb768103000 tx=0x7fb758005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:37.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.753+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb760003c60 con 0x7fb768103b60 2024-04-25T13:04:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.940+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb768062b50 con 0x7fb768103b60 2024-04-25T13:04:37.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.940+0000 7fb76ca03640 1 -- 172.21.15.38:0/2860171675 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb76009cb50 con 0x7fb768103b60 2024-04-25T13:04:37.939 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 msgr2=0x7fb7500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb768103000 tx=0x7fb758005f90 comp rx=0 tx=0).stop 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 msgr2=0x7fb768074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 0x7fb768074880 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb760009fd0 tx=0x7fb76000b260 comp rx=0 tx=0).stop 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 shutdown_connections 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fb7500a3bb0 0x7fb7500a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768103b60 0x7fb768074880 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7681031a0 0x7fb768074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 --2- 172.21.15.38:0/2860171675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb768101fb0 0x7fb768075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 >> 172.21.15.38:0/2860171675 conn(0x7fb7680fd520 msgr2=0x7fb768106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 shutdown_connections 2024-04-25T13:04:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:37.942+0000 7fb76da05640 1 -- 172.21.15.38:0/2860171675 wait complete. 2024-04-25T13:04:37.954 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 -- 172.21.15.38:0/2285695895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0107c50 msgr2=0x7effc010a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 --2- 172.21.15.38:0/2285695895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0107c50 0x7effc010a0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7effb8009900 tx=0x7effb802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 -- 172.21.15.38:0/2285695895 shutdown_connections 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 --2- 172.21.15.38:0/2285695895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0107c50 0x7effc010a0a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 --2- 172.21.15.38:0/2285695895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0105320 0x7effc0107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 --2- 172.21.15.38:0/2285695895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 0x7effc0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.060+0000 7effc738e640 1 -- 172.21.15.38:0/2285695895 >> 172.21.15.38:0/2285695895 conn(0x7effc00fc3e0 msgr2=0x7effc00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 -- 172.21.15.38:0/2285695895 shutdown_connections 2024-04-25T13:04:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 -- 172.21.15.38:0/2285695895 wait complete. 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 Processor -- start 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 -- start start 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 0x7effc0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 0x7effc0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0107c50 0x7effc019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7effc010dc50 con 0x7effc0105320 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7effc010dad0 con 0x7effc0107c50 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc738e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effc010ddd0 con 0x7effc01029f0 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc4929640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 0x7effc0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc4929640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 0x7effc0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47470/0 (socket says 172.21.15.38:47470) 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.061+0000 7effc4929640 1 -- 172.21.15.38:0/1439091496 learned_addr learned my addr 172.21.15.38:0/1439091496 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc512a640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 0x7effc0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc592b640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0107c50 0x7effc019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc4929640 1 -- 172.21.15.38:0/1439091496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 msgr2=0x7effc0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc4929640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 0x7effc0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc4929640 1 -- 172.21.15.38:0/1439091496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0107c50 msgr2=0x7effc019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc4929640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0107c50 0x7effc019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc4929640 1 -- 172.21.15.38:0/1439091496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc019fe50 con 0x7effc0105320 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc592b640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0107c50 0x7effc019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc512a640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 0x7effc0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc4929640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 0x7effc0198770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7effb000d770 tx=0x7effb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0017070 con 0x7effc0105320 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7effb0009c30 con 0x7effc0105320 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc01a0140 con 0x7effc0105320 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb000f620 con 0x7effc0105320 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.062+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01a0690 con 0x7effc0105320 2024-04-25T13:04:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.063+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc00fded0 con 0x7effc0105320 2024-04-25T13:04:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.064+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7effb001a070 con 0x7effc0105320 2024-04-25T13:04:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.065+0000 7effb67fc640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7effac0a3bb0 0x7effac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.065+0000 7effc512a640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7effac0a3bb0 0x7effac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.065+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7effb00cff70 con 0x7effc0105320 2024-04-25T13:04:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.065+0000 7effc512a640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7effac0a3bb0 0x7effac0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7effbc005fd0 tx=0x7effbc009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:38.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.068+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effb0009de0 con 0x7effc0105320 2024-04-25T13:04:38.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.254+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7effc002d810 con 0x7effc0105320 2024-04-25T13:04:38.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.255+0000 7effb67fc640 1 -- 172.21.15.38:0/1439091496 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7effb009c950 con 0x7effc0105320 2024-04-25T13:04:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T13:04:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7effac0a3bb0 msgr2=0x7effac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7effac0a3bb0 0x7effac0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7effbc005fd0 tx=0x7effbc009520 comp rx=0 tx=0).stop 2024-04-25T13:04:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 msgr2=0x7effc0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 0x7effc0198770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7effb000d770 tx=0x7effb000dc30 comp rx=0 tx=0).stop 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 shutdown_connections 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7effac0a3bb0 0x7effac0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effc0107c50 0x7effc019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc0105320 0x7effc0198770 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 --2- 172.21.15.38:0/1439091496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc01029f0 0x7effc0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 >> 172.21.15.38:0/1439091496 conn(0x7effc00fc3e0 msgr2=0x7effc0103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 shutdown_connections 2024-04-25T13:04:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.257+0000 7effc738e640 1 -- 172.21.15.38:0/1439091496 wait complete. 2024-04-25T13:04:38.265 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:38 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2069497202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:38.265 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:38 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/4024287127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:38.265 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:38 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2860171675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:38.266 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:04:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 -- 172.21.15.38:0/2491321533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350074930 msgr2=0x7f3350074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 --2- 172.21.15.38:0/2491321533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350074930 0x7f3350074d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3338009900 tx=0x7f333802a480 comp rx=0 tx=0).stop 2024-04-25T13:04:38.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 -- 172.21.15.38:0/2491321533 shutdown_connections 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 --2- 172.21.15.38:0/2491321533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3350109e60 0x7f335010c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 --2- 172.21.15.38:0/2491321533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350074930 0x7f3350074d90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 --2- 172.21.15.38:0/2491321533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3350075f10 0x7f3350074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 -- 172.21.15.38:0/2491321533 >> 172.21.15.38:0/2491321533 conn(0x7f33500fd520 msgr2=0x7f33500ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 -- 172.21.15.38:0/2491321533 shutdown_connections 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.372+0000 7f3356368640 1 -- 172.21.15.38:0/2491321533 wait complete. 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 Processor -- start 2024-04-25T13:04:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 -- start start 2024-04-25T13:04:38.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3350074930 0x7f335019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3350075f10 0x7f335019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 0x7f33501a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fe20 con 0x7f3350109e60 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fca0 con 0x7f3350075f10 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3356368640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f335010ffa0 con 0x7f3350074930 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3354905640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 0x7f33501a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3354905640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 0x7f33501a1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47486/0 (socket says 172.21.15.38:47486) 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f3354905640 1 -- 172.21.15.38:0/2362784298 learned_addr learned my addr 172.21.15.38:0/2362784298 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.373+0000 7f334f7fe640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3350075f10 0x7f335019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f334ffff640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3350074930 0x7f335019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3354905640 1 -- 172.21.15.38:0/2362784298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3350074930 msgr2=0x7f335019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3354905640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3350074930 0x7f335019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3354905640 1 -- 172.21.15.38:0/2362784298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3350075f10 msgr2=0x7f335019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3354905640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3350075f10 0x7f335019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3354905640 1 -- 172.21.15.38:0/2362784298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33501a2100 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3354905640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 0x7f33501a1a90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f334800cad0 tx=0x7f334800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334800ae20 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3348005020 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33501a23f0 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334800f6b0 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.374+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33501a28c0 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.375+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3350108c20 con 0x7f3350109e60 2024-04-25T13:04:38.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.376+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f33480051d0 con 0x7f3350109e60 2024-04-25T13:04:38.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.377+0000 7f334d7fa640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f333c0a3bb0 0x7f333c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.377+0000 7f334ffff640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f333c0a3bb0 0x7f333c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.377+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f33480d0500 con 0x7f3350109e60 2024-04-25T13:04:38.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.377+0000 7f334ffff640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f333c0a3bb0 0x7f333c0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3340005fd0 tx=0x7f3340005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:38.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.381+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3348014030 con 0x7f3350109e60 2024-04-25T13:04:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:38 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2069497202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:38 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/4024287127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:38 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2860171675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.568+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3350062b50 con 0x7f3350109e60 2024-04-25T13:04:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.568+0000 7f334d7fa640 1 -- 172.21.15.38:0/2362784298 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f334809cea0 con 0x7f3350109e60 2024-04-25T13:04:38.567 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T13:04:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.570+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f333c0a3bb0 msgr2=0x7f333c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.570+0000 7f3356368640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f333c0a3bb0 0x7f333c0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3340005fd0 tx=0x7f3340005e20 comp rx=0 tx=0).stop 2024-04-25T13:04:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 msgr2=0x7f33501a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 0x7f33501a1a90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f334800cad0 tx=0x7f334800cf90 comp rx=0 tx=0).stop 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 shutdown_connections 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f333c0a3bb0 0x7f333c0a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3350109e60 0x7f33501a1a90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3350075f10 0x7f335019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 --2- 172.21.15.38:0/2362784298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3350074930 0x7f335019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 >> 172.21.15.38:0/2362784298 conn(0x7f33500fd520 msgr2=0x7f3350105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 shutdown_connections 2024-04-25T13:04:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.571+0000 7f3356368640 1 -- 172.21.15.38:0/2362784298 wait complete. 2024-04-25T13:04:38.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:38 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2069497202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:38.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:38 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/4024287127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:38.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:38 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2860171675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:04:38.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:04:38.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 -- 172.21.15.38:0/229273152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac101fc0 msgr2=0x7fbbac1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 --2- 172.21.15.38:0/229273152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac101fc0 0x7fbbac1023e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 -- 172.21.15.38:0/229273152 shutdown_connections 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 --2- 172.21.15.38:0/229273152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac103b70 0x7fbbac10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 --2- 172.21.15.38:0/229273152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 0x7fbbac103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 --2- 172.21.15.38:0/229273152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac101fc0 0x7fbbac1023e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 -- 172.21.15.38:0/229273152 >> 172.21.15.38:0/229273152 conn(0x7fbbac0fd550 msgr2=0x7fbbac0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 -- 172.21.15.38:0/229273152 shutdown_connections 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 -- 172.21.15.38:0/229273152 wait complete. 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 Processor -- start 2024-04-25T13:04:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.684+0000 7fbbb2410640 1 -- start start 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac101fc0 0x7fbbac1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 0x7fbbac1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 0x7fbbac19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac10dc70 con 0x7fbbac103b70 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac10daf0 con 0x7fbbac1031b0 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac10ddf0 con 0x7fbbac101fc0 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 0x7fbbac19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 0x7fbbac19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47496/0 (socket says 172.21.15.38:47496) 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 -- 172.21.15.38:0/187223911 learned_addr learned my addr 172.21.15.38:0/187223911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbabfff640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac101fc0 0x7fbbac1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbab7fe640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 0x7fbbac1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 -- 172.21.15.38:0/187223911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac101fc0 msgr2=0x7fbbac1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac101fc0 0x7fbbac1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 -- 172.21.15.38:0/187223911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 msgr2=0x7fbbac1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 0x7fbbac1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 -- 172.21.15.38:0/187223911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19fec0 con 0x7fbbac103b70 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbabfff640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac101fc0 0x7fbbac1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbab7fe640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 0x7fbbac1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb09ad640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 0x7fbbac19f850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbba400eaa0 tx=0x7fbba400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cdb0 con 0x7fbbac103b70 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac1a01b0 con 0x7fbbac103b70 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbba4005020 con 0x7fbbac103b70 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba40116d0 con 0x7fbbac103b70 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.685+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a06b0 con 0x7fbbac103b70 2024-04-25T13:04:38.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.686+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbac1092e0 con 0x7fbbac103b70 2024-04-25T13:04:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.688+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbba40051d0 con 0x7fbbac103b70 2024-04-25T13:04:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.688+0000 7fbba97fa640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbb940a3bb0 0x7fbb940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.688+0000 7fbbabfff640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbb940a3bb0 0x7fbb940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.688+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbba40d0ab0 con 0x7fbbac103b70 2024-04-25T13:04:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.689+0000 7fbbabfff640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbb940a3bb0 0x7fbb940a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006010 tx=0x7fbb9c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.692+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbba4016030 con 0x7fbbac103b70 2024-04-25T13:04:38.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.917+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbac02d5c0 con 0x7fbbac103b70 2024-04-25T13:04:38.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.918+0000 7fbba97fa640 1 -- 172.21.15.38:0/187223911 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11938 (secure 0 0 0) 0x7fbba409d490 con 0x7fbbac103b70 2024-04-25T13:04:38.916 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:38.917 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:02:51.699933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24484,24518,24482,24466,14625]},"id":1}]} 2024-04-25T13:04:38.917 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-04-25T13:04:38.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbb940a3bb0 msgr2=0x7fbb940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbb940a3bb0 0x7fbb940a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006010 tx=0x7fbb9c031510 comp rx=0 tx=0).stop 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 msgr2=0x7fbbac19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 0x7fbbac19f850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbba400eaa0 tx=0x7fbba400ef60 comp rx=0 tx=0).stop 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 shutdown_connections 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fbb940a3bb0 0x7fbb940a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbac103b70 0x7fbbac19f850 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbbac1031b0 0x7fbbac1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 --2- 172.21.15.38:0/187223911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac101fc0 0x7fbbac1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 >> 172.21.15.38:0/187223911 conn(0x7fbbac0fd550 msgr2=0x7fbbac1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 shutdown_connections 2024-04-25T13:04:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:38.920+0000 7fbbb2410640 1 -- 172.21.15.38:0/187223911 wait complete. 2024-04-25T13:04:38.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 -- 172.21.15.38:0/730958356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0103190 msgr2=0x7f09d0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 --2- 172.21.15.38:0/730958356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0103190 0x7f09d0103610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009900 tx=0x7f09bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 -- 172.21.15.38:0/730958356 shutdown_connections 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 --2- 172.21.15.38:0/730958356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103b50 0x7f09d010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 --2- 172.21.15.38:0/730958356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0103190 0x7f09d0103610 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 --2- 172.21.15.38:0/730958356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0101fa0 0x7f09d01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 -- 172.21.15.38:0/730958356 >> 172.21.15.38:0/730958356 conn(0x7f09d00fd530 msgr2=0x7f09d00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 -- 172.21.15.38:0/730958356 shutdown_connections 2024-04-25T13:04:39.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.034+0000 7f09d719b640 1 -- 172.21.15.38:0/730958356 wait complete. 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 Processor -- start 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 -- start start 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 0x7f09d0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103190 0x7f09d0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0103b50 0x7f09d0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09d010dc50 con 0x7f09d0101fa0 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09d010dad0 con 0x7f09d0103190 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d719b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09d010ddd0 con 0x7f09d0103b50 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 0x7f09d0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 0x7f09d0075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47516/0 (socket says 172.21.15.38:47516) 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 -- 172.21.15.38:0/3784836780 learned_addr learned my addr 172.21.15.38:0/3784836780 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09cffff640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103190 0x7f09d0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d5738640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0103b50 0x7f09d0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 -- 172.21.15.38:0/3784836780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0103b50 msgr2=0x7f09d0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0103b50 0x7f09d0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 -- 172.21.15.38:0/3784836780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103190 msgr2=0x7f09d0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103190 0x7f09d0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 -- 172.21.15.38:0/3784836780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d0075110 con 0x7f09d0101fa0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d5738640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0103b50 0x7f09d0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.035+0000 7f09d4f37640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 0x7f09d0075d00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f09c400d770 tx=0x7f09c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.036+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4017070 con 0x7f09d0101fa0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.036+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f09c4009c30 con 0x7f09d0101fa0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.036+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d01a2390 con 0x7f09d0101fa0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.036+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4003d70 con 0x7f09d0101fa0 2024-04-25T13:04:39.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.036+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01a2900 con 0x7f09d0101fa0 2024-04-25T13:04:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.037+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d00691c0 con 0x7f09d0101fa0 2024-04-25T13:04:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.038+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f09c4003f10 con 0x7f09d0101fa0 2024-04-25T13:04:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.038+0000 7f09cdffb640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f09c00a3c30 0x7f09c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.039+0000 7f09cffff640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f09c00a3c30 0x7f09c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.039+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f09c40cfd10 con 0x7f09d0101fa0 2024-04-25T13:04:39.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.039+0000 7f09cffff640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f09c00a3c30 0x7f09c00a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09bc031570 tx=0x7f09bc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.043+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09c409c6f0 con 0x7f09d0101fa0 2024-04-25T13:04:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.233+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09d0062d10 con 0x7f09d0101fa0 2024-04-25T13:04:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.235+0000 7f09cdffb640 1 -- 172.21.15.38:0/3784836780 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22650 (secure 0 0 0) 0x7f09c40bc070 con 0x7f09d0101fa0 2024-04-25T13:04:39.234 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:04:39.234 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:03:37.115660+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f09c00a3c30 msgr2=0x7f09c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f09c00a3c30 0x7f09c00a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09bc031570 tx=0x7f09bc005a20 comp rx=0 tx=0).stop 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 msgr2=0x7f09d0075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 0x7f09d0075d00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f09c400d770 tx=0x7f09c400dc30 comp rx=0 tx=0).stop 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 shutdown_connections 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f09c00a3c30 0x7f09c00a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0103b50 0x7f09d0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103190 0x7f09d0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 --2- 172.21.15.38:0/3784836780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d0101fa0 0x7f09d0075d00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 >> 172.21.15.38:0/3784836780 conn(0x7f09d00fd530 msgr2=0x7f09d0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 shutdown_connections 2024-04-25T13:04:39.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.237+0000 7f09d719b640 1 -- 172.21.15.38:0/3784836780 wait complete. 2024-04-25T13:04:39.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:02:44.261324+0000', 'modified': '2024-04-25T13:02:51.699933+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24518, 'mds_1': 24484, 'mds_2': 24482, 'mds_3': 24466, 'mds_4': 14625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/3427364907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 3427364907}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 3427364907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.38:6841/4187315479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 4187315479}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 4187315479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/2494647672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 2494647672}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 2494647672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/650369302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 650369302}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 650369302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/4172911078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 4172911078}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 4172911078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/2362992725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 2362992725}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2362992725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.178:6837/2053792208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2053792208}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2053792208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.104:6837/2247876378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2247876378}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2247876378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.178:6839/391098246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 391098246}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 391098246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/1961595450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1961595450}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1961595450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24484, 24518, 24482, 24466, 14625]} 2024-04-25T13:04:39.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T13:04:39.248 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:04:39.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1692115154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0101fe0 msgr2=0x7ff0c0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1692115154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0101fe0 0x7ff0c0102460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009900 tx=0x7ff0b402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1692115154 shutdown_connections 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1692115154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 0x7ff0c01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1692115154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0101fe0 0x7ff0c0102460 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1692115154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0c0100e90 0x7ff0c01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1692115154 >> 172.21.15.38:0/1692115154 conn(0x7ff0c00fc400 msgr2=0x7ff0c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1692115154 shutdown_connections 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.351+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1692115154 wait complete. 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 Processor -- start 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 -- start start 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 0x7ff0c01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0c0101fe0 0x7ff0c0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 0x7ff0c019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c010dc80 con 0x7ff0c0100e90 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c010db00 con 0x7ff0c0101fe0 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0c5d3d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c010de00 con 0x7ff0c01029a0 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 0x7ff0c01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 0x7ff0c01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47534/0 (socket says 172.21.15.38:47534) 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.352+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/1327268640 learned_addr learned my addr 172.21.15.38:0/1327268640 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0beffd640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0c0101fe0 0x7ff0c0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bffff640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 0x7ff0c019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/1327268640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 msgr2=0x7ff0c019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 0x7ff0c019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/1327268640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0c0101fe0 msgr2=0x7ff0c0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0c0101fe0 0x7ff0c0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/1327268640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c019ff10 con 0x7ff0c0100e90 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bffff640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 0x7ff0c019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 0x7ff0c01982f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000cce0 tx=0x7ff0b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bcff9640 1 -- 172.21.15.38:0/1327268640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0004b20 con 0x7ff0c0100e90 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c01a0200 con 0x7ff0c0100e90 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bcff9640 1 -- 172.21.15.38:0/1327268640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0b0004cc0 con 0x7ff0c0100e90 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0bcff9640 1 -- 172.21.15.38:0/1327268640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000f740 con 0x7ff0c0100e90 2024-04-25T13:04:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.353+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c01a07a0 con 0x7ff0c0100e90 2024-04-25T13:04:39.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0bcff9640 1 -- 172.21.15.38:0/1327268640 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff0b0005580 con 0x7ff0c0100e90 2024-04-25T13:04:39.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 Processor -- start 2024-04-25T13:04:39.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 -- start start 2024-04-25T13:04:39.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d718b80 0x555e8d718fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 0x555e8d71a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d720fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e8d67dc50 con 0x555e8d719d50 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e8d61a830 con 0x555e8d718b80 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.356+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555e8d679c10 con 0x555e8d71a710 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 0x555e8d71a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 0x555e8d71a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47538/0 (socket says 172.21.15.38:47538) 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2503760363 learned_addr learned my addr 172.21.15.38:0/2503760363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d720fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bffff640 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d718b80 0x555e8d718fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bcff9640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0a80a3bb0 0x7ff0a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bffff640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0a80a3bb0 0x7ff0a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2503760363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 msgr2=0x555e8d720fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d720fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2503760363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d718b80 msgr2=0x555e8d718fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d718b80 0x555e8d718fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bcff9640 1 -- 172.21.15.38:0/1327268640 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff0b00d0490 con 0x7ff0c0100e90 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2503760363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8d3902d0 con 0x555e8d719d50 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 0x555e8d71a1d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0098bd0 tx=0x7ff0b00519b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79379f9280bf666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0bffff640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0a80a3bb0 0x7ff0a80a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x555e8d719c50 tx=0x7ff0b80046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0a5ffb640 1 -- 172.21.15.38:0/2503760363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001d030 con 0x555e8d719d50 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0a5ffb640 1 -- 172.21.15.38:0/2503760363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0b009ce70 con 0x555e8d719d50 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.357+0000 7ff0a5ffb640 1 -- 172.21.15.38:0/2503760363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00a53e0 con 0x555e8d719d50 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2503760363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 msgr2=0x555e8d71a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 0x555e8d71a1d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0098bd0 tx=0x7ff0b00519b0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2503760363 shutdown_connections 2024-04-25T13:04:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d720fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d719d50 0x555e8d71a1d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2503760363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d718b80 0x555e8d718fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2503760363 >> 172.21.15.38:0/2503760363 conn(0x555e8d714070 msgr2=0x555e8d714470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2503760363 shutdown_connections 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2503760363 wait complete. 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.358+0000 7ff0c8a4b740 1 -- start start 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 Processor -- start 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d722f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d719d50 0x555e8d723480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d71a710 0x555e8d72a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e8d7216c0 con 0x555e8d718b80 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e8d618470 con 0x555e8d71a710 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555e8d721840 con 0x555e8d719d50 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0c8a4b740 10 client.0 _ll_register_callbacks cb 0x555e8d63df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d722f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d722f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47542/0 (socket says 172.21.15.38:47542) 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2732796869 learned_addr learned my addr 172.21.15.38:0/2732796869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d719d50 0x555e8d723480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2732796869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d719d50 msgr2=0x555e8d723480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d719d50 0x555e8d723480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2732796869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d71a710 msgr2=0x555e8d72a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d71a710 0x555e8d72a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2732796869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8d45a6b0 con 0x555e8d718b80 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d719d50 0x555e8d723480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d722f40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff0b009c700 tx=0x7ff0b009c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8976c7038466917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff09effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.359+0000 7ff09effd640 1 -- 172.21.15.38:0/2732796869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001d050 con 0x555e8d718b80 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff09effd640 1 -- 172.21.15.38:0/2732796869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff0b0098de0 con 0x555e8d718b80 2024-04-25T13:04:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b009ce70 con 0x555e8d718b80 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff09effd640 1 -- 172.21.15.38:0/2732796869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0b00a53e0 con 0x555e8d718b80 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 10 client.15243 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e8d72d830 con 0x555e8d718b80 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff09effd640 1 -- 172.21.15.38:0/2732796869 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0b009ce70 con 0x555e8d718b80 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff09effd640 1 -- 172.21.15.38:0/2732796869 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0b0051b40 con 0x555e8d718b80 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 10 client.15243 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 20 client.15243 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d72fa60 0x555e8d730ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 4 client.15243 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 --> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e8d68b670 con 0x555e8d72fa60 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.360+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d72fa60 0x555e8d730ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff09effd640 10 client.15243 ms_handle_connect on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d72fa60 0x555e8d730ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bd98b6f5f3dc18b server_cookie=84cafff843ac3971 in_seq=0 out_seq=0 2024-04-25T13:04:39.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff09effd640 1 -- 172.21.15.38:0/2732796869 <== mds.4 v2:172.21.15.38:6840/4187315479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e8d68b670 con 0x555e8d72fa60 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff09effd640 10 client.15243 handle_command_reply: tid=0 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 client.15243 shutdown 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 msgr2=0x555e8d722f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d722f40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff0b009c700 tx=0x7ff0b009c320 comp rx=0 tx=0).stop 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 shutdown_connections 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d72fa60 0x555e8d730ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d71a710 0x555e8d72a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d719d50 0x555e8d723480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2732796869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d722f40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff09effd640 0 client.15243 ms_handle_reset on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 >> 172.21.15.38:0/2732796869 conn(0x555e8d714070 msgr2=0x555e8d7131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 shutdown_connections 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.361+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2732796869 wait complete. 2024-04-25T13:04:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.362+0000 7ff0c8a4b740 20 client.15243 trim_cache size 0 max 16384 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 Processor -- start 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 -- start start 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d720e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d72a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d71d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e8d72f210 con 0x555e8d718b80 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e8d72f8e0 con 0x555e8d719d50 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555e8d72ffb0 con 0x555e8d71a710 2024-04-25T13:04:39.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d720e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d720e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47554/0 (socket says 172.21.15.38:47554) 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/3385261125 learned_addr learned my addr 172.21.15.38:0/3385261125 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bffff640 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d71d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/3385261125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 msgr2=0x555e8d71d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d71d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/3385261125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 msgr2=0x555e8d72a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d72a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/3385261125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8d7197d0 con 0x555e8d718b80 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d720e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00a55a0 tx=0x7ff0b009ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40575f2a4ef548bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.380+0000 7ff0a57fa640 1 -- 172.21.15.38:0/3385261125 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001d070 con 0x555e8d718b80 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0a57fa640 1 -- 172.21.15.38:0/3385261125 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0b009ce70 con 0x555e8d718b80 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0a57fa640 1 -- 172.21.15.38:0/3385261125 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0098de0 con 0x555e8d718b80 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/3385261125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 msgr2=0x555e8d720e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d720e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00a55a0 tx=0x7ff0b009ac80 comp rx=0 tx=0).stop 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/3385261125 shutdown_connections 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d71d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d72a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/3385261125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d720e20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/3385261125 >> 172.21.15.38:0/3385261125 conn(0x555e8d714070 msgr2=0x555e8d714450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/3385261125 shutdown_connections 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/3385261125 wait complete. 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.381+0000 7ff0c8a4b740 1 -- start start 2024-04-25T13:04:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 Processor -- start 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d7254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d725a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d793a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e8d730d50 con 0x555e8d718b80 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e8d730680 con 0x555e8d719d50 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555e8d731420 con 0x555e8d71a710 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0c8a4b740 10 client.0 _ll_register_callbacks cb 0x7ff0b800b1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d7254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d7254e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47558/0 (socket says 172.21.15.38:47558) 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2105093393 learned_addr learned my addr 172.21.15.38:0/2105093393 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bffff640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d793a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d725a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2105093393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 msgr2=0x555e8d793a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d793a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2105093393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 msgr2=0x555e8d725a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d725a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 -- 172.21.15.38:0/2105093393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8d793f90 con 0x555e8d718b80 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d725a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff0bf7fe640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d7254e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00a4eb0 tx=0x7ff0b009c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff12cee85b54a07c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.382+0000 7ff09effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff09effd640 1 -- 172.21.15.38:0/2105093393 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001d070 con 0x555e8d718b80 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff09effd640 1 -- 172.21.15.38:0/2105093393 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff0b0051b40 con 0x555e8d718b80 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b009ce70 con 0x555e8d718b80 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff09effd640 1 -- 172.21.15.38:0/2105093393 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0b0051d40 con 0x555e8d718b80 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 10 client.15249 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e8d795290 con 0x555e8d718b80 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff09effd640 1 -- 172.21.15.38:0/2105093393 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0b009ce70 con 0x555e8d718b80 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff09effd640 1 -- 172.21.15.38:0/2105093393 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0b0005000 con 0x555e8d718b80 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 10 client.15249 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 10 client.15249 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 20 client.15249 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d7998d0 0x555e8d7985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 4 client.15249 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 --> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] -- command(tid 0: {"prefix": "status"}) -- 0x555e8d6854e0 con 0x555e8d7998d0 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.383+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d7998d0 0x555e8d7985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff09effd640 10 client.15249 ms_handle_connect on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0beffd640 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d7998d0 0x555e8d7985e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11704c4c81818727 server_cookie=252f43b76f5122fc in_seq=0 out_seq=0 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff09effd640 1 -- 172.21.15.38:0/2105093393 <== mds.4 v2:172.21.15.38:6840/4187315479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555e8d6854e0 con 0x555e8d7998d0 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff09effd640 10 client.15249 handle_command_reply: tid=0 2024-04-25T13:04:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 client.15249 shutdown 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 msgr2=0x555e8d7254e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d7254e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00a4eb0 tx=0x7ff0b009c2b0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 shutdown_connections 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:6840/4187315479,v1:172.21.15.38:6841/4187315479] conn(0x555e8d7998d0 0x555e8d7985e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e8d71a710 0x555e8d793a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff09effd640 0 client.15249 ms_handle_reset on v2:172.21.15.38:6840/4187315479 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e8d719d50 0x555e8d725a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 --2- 172.21.15.38:0/2105093393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8d718b80 0x555e8d7254e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 >> 172.21.15.38:0/2105093393 conn(0x555e8d714070 msgr2=0x555e8d71d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 shutdown_connections 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 1 -- 172.21.15.38:0/2105093393 wait complete. 2024-04-25T13:04:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.384+0000 7ff0c8a4b740 20 client.15249 trim_cache size 0 max 16384 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 14625, 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 108.692653158, 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:39.390 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:39.391 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:39.391 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 121.65199388800001 2024-04-25T13:04:39.391 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:39.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0a80a3bb0 msgr2=0x7ff0a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0a80a3bb0 0x7ff0a80a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x555e8d719c50 tx=0x7ff0b80046c0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 msgr2=0x7ff0c01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 0x7ff0c01982f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000cce0 tx=0x7ff0b000a620 comp rx=0 tx=0).stop 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 shutdown_connections 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff0a80a3bb0 0x7ff0a80a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0c01029a0 0x7ff0c019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0c0101fe0 0x7ff0c0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 --2- 172.21.15.38:0/1327268640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0c0100e90 0x7ff0c01982f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 >> 172.21.15.38:0/1327268640 conn(0x7ff0c00fc400 msgr2=0x7ff0c0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 shutdown_connections 2024-04-25T13:04:39.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.385+0000 7ff0c5d3d640 1 -- 172.21.15.38:0/1327268640 wait complete. 2024-04-25T13:04:39.394 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:04:39.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:39 smithi178 ceph-mon[33233]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:39.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:39 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1439091496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:39.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:39 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2362784298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:39.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:39 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/187223911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:39.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.498+0000 7f8a63bad640 1 -- 172.21.15.38:0/4031995519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 msgr2=0x7f8a5c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.498+0000 7f8a63bad640 1 --2- 172.21.15.38:0/4031995519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c10a0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009ca0 tx=0x7f8a5402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- 172.21.15.38:0/4031995519 shutdown_connections 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 --2- 172.21.15.38:0/4031995519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c10a0e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 --2- 172.21.15.38:0/4031995519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c1053c0 0x7f8a5c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 --2- 172.21.15.38:0/4031995519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a5c102a90 0x7f8a5c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- 172.21.15.38:0/4031995519 >> 172.21.15.38:0/4031995519 conn(0x7f8a5c0fc440 msgr2=0x7f8a5c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- 172.21.15.38:0/4031995519 shutdown_connections 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- 172.21.15.38:0/4031995519 wait complete. 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 Processor -- start 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- start start 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a5c102a90 0x7f8a5c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c1053c0 0x7f8a5c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10dc30 con 0x7f8a5c107cf0 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10dab0 con 0x7f8a5c1053c0 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.499+0000 7f8a63bad640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10ddb0 con 0x7f8a5c102a90 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47576/0 (socket says 172.21.15.38:47576) 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 -- 172.21.15.38:0/2588463864 learned_addr learned my addr 172.21.15.38:0/2588463864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a61148640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c1053c0 0x7f8a5c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a61949640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a5c102a90 0x7f8a5c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 -- 172.21.15.38:0/2588463864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a5c102a90 msgr2=0x7f8a5c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a5c102a90 0x7f8a5c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 -- 172.21.15.38:0/2588463864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c1053c0 msgr2=0x7f8a5c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c1053c0 0x7f8a5c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 -- 172.21.15.38:0/2588463864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c19fe70 con 0x7f8a5c107cf0 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a6214a640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c19f800 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009dc0 tx=0x7f8a54031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a52ffd640 1 -- 172.21.15.38:0/2588463864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54003bf0 con 0x7f8a5c107cf0 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a52ffd640 1 -- 172.21.15.38:0/2588463864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8a54003d90 con 0x7f8a5c107cf0 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c1a0100 con 0x7f8a5c107cf0 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a52ffd640 1 -- 172.21.15.38:0/2588463864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5402fcd0 con 0x7f8a5c107cf0 2024-04-25T13:04:39.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.500+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1a0650 con 0x7f8a5c107cf0 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a52ffd640 1 -- 172.21.15.38:0/2588463864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a54004040 con 0x7f8a5c107cf0 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 Processor -- start 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 -- start start 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea5f90 0x55a7b1ea63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1ea75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1e06c50 con 0x55a7b1ea7b20 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1da3830 con 0x55a7b1ea5f90 2024-04-25T13:04:39.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.503+0000 7f8a669a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1e02c10 con 0x55a7b1ea7160 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47592/0 (socket says 172.21.15.38:47592) 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 -- 172.21.15.38:0/2948674209 learned_addr learned my addr 172.21.15.38:0/2948674209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a52ffd640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8a480a3bb0 0x7f8a480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea5f90 0x55a7b1ea63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61949640 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1ea75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a6214a640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8a480a3bb0 0x7f8a480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 -- 172.21.15.38:0/2948674209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 msgr2=0x55a7b1ea75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1ea75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 -- 172.21.15.38:0/2948674209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea5f90 msgr2=0x55a7b1ea63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea5f90 0x55a7b1ea63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a52ffd640 1 -- 172.21.15.38:0/2588463864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8a540f1ab0 con 0x7f8a5c107cf0 2024-04-25T13:04:39.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 -- 172.21.15.38:0/2948674209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b1b192d0 con 0x55a7b1ea7b20 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a61148640 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eae3b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009fd0 tx=0x7f8a4c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36196d5aa64fa673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a47fff640 1 -- 172.21.15.38:0/2948674209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c004030 con 0x55a7b1ea7b20 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a6214a640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8a480a3bb0 0x7f8a480a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a540ba1f0 tx=0x7f8a540de040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a47fff640 1 -- 172.21.15.38:0/2948674209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8a4c00fc60 con 0x55a7b1ea7b20 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.504+0000 7f8a47fff640 1 -- 172.21.15.38:0/2948674209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c003a10 con 0x55a7b1ea7b20 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 -- 172.21.15.38:0/2948674209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 msgr2=0x55a7b1eae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eae3b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009fd0 tx=0x7f8a4c005850 comp rx=0 tx=0).stop 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 -- 172.21.15.38:0/2948674209 shutdown_connections 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eae3b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1ea75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 --2- 172.21.15.38:0/2948674209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea5f90 0x55a7b1ea63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 -- 172.21.15.38:0/2948674209 >> 172.21.15.38:0/2948674209 conn(0x55a7b1e9d070 msgr2=0x55a7b1e9d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 -- 172.21.15.38:0/2948674209 shutdown_connections 2024-04-25T13:04:39.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 -- 172.21.15.38:0/2948674209 wait complete. 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.505+0000 7f8a669a7740 1 -- start start 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 Processor -- start 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1eaeb30 con 0x55a7b1ea7160 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1da1470 con 0x55a7b1ea7b20 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1eaecb0 con 0x55a7b1ea5f90 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a669a7740 10 client.0 _ll_register_callbacks cb 0x55a7b1dc6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47608/0 (socket says 172.21.15.38:47608) 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 -- 172.21.15.38:0/3162843511 learned_addr learned my addr 172.21.15.38:0/3162843511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61148640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61148640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 -- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea5f90 msgr2=0x55a7b1eb03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 -- 172.21.15.38:0/3162843511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 msgr2=0x55a7b1eb7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 -- 172.21.15.38:0/3162843511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b1be36b0 con 0x55a7b1ea7160 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a61949640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800bf80 tx=0x7f8a580128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0c2ebf03740c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.506+0000 7f8a457fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a457fa640 1 -- 172.21.15.38:0/3162843511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5801f060 con 0x55a7b1ea7160 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a457fa640 1 -- 172.21.15.38:0/3162843511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8a580108a0 con 0x55a7b1ea7160 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c00fc60 con 0x55a7b1ea7160 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a457fa640 1 -- 172.21.15.38:0/3162843511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a58010aa0 con 0x55a7b1ea7160 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 10 client.15261 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a7b1ebada0 con 0x55a7b1ea7160 2024-04-25T13:04:39.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a457fa640 1 -- 172.21.15.38:0/3162843511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8a4c00fc60 con 0x55a7b1ea7160 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a457fa640 1 -- 172.21.15.38:0/3162843511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a58024070 con 0x55a7b1ea7160 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 20 client.15261 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1ebd490 0x55a7b1ebe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 4 client.15261 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.507+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 --> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7b1e14670 con 0x55a7b1ebd490 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.508+0000 7f8a61148640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1ebd490 0x55a7b1ebe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.508+0000 7f8a61148640 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1ebd490 0x55a7b1ebe900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96505ff8145a0b6a server_cookie=68ad3da368a02f61 in_seq=0 out_seq=0 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.508+0000 7f8a457fa640 10 client.15261 ms_handle_connect on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a457fa640 1 -- 172.21.15.38:0/3162843511 <== mds.3 v2:172.21.15.178:6836/2053792208 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7b1e14670 con 0x55a7b1ebd490 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a457fa640 10 client.15261 handle_command_reply: tid=0 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 client.15261 shutdown 2024-04-25T13:04:39.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 msgr2=0x55a7b1eb0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800bf80 tx=0x7f8a580128e0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 shutdown_connections 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1ebd490 0x55a7b1ebe900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a457fa640 0 client.15261 ms_handle_reset on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3162843511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 >> 172.21.15.38:0/3162843511 conn(0x55a7b1e9d070 msgr2=0x55a7b1e9c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 shutdown_connections 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 1 -- 172.21.15.38:0/3162843511 wait complete. 2024-04-25T13:04:39.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.509+0000 7f8a669a7740 20 client.15261 trim_cache size 0 max 16384 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.527+0000 7f8a669a7740 1 Processor -- start 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.527+0000 7f8a669a7740 1 -- start start 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.527+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.527+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.527+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a669a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1ebcc40 con 0x55a7b1ea5f90 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a669a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1ebd310 con 0x55a7b1ea7b20 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a669a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1ebd9e0 con 0x55a7b1ea7160 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47622/0 (socket says 172.21.15.38:47622) 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 -- 172.21.15.38:0/4280160809 learned_addr learned my addr 172.21.15.38:0/4280160809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 -- 172.21.15.38:0/4280160809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 msgr2=0x55a7b1eb0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61949640 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 -- 172.21.15.38:0/4280160809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 msgr2=0x55a7b1eb7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 -- 172.21.15.38:0/4280160809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b1e9cd40 con 0x55a7b1ea5f90 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61949640 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a61148640 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009560 tx=0x7f8a4c0115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ca21e703c8b688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a477fe640 1 -- 172.21.15.38:0/4280160809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c031070 con 0x55a7b1ea5f90 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a477fe640 1 -- 172.21.15.38:0/4280160809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8a4c00fc60 con 0x55a7b1ea5f90 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.528+0000 7f8a477fe640 1 -- 172.21.15.38:0/4280160809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c02b0d0 con 0x55a7b1ea5f90 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- 172.21.15.38:0/4280160809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 msgr2=0x55a7b1eb03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009560 tx=0x7f8a4c0115f0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- 172.21.15.38:0/4280160809 shutdown_connections 2024-04-25T13:04:39.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- 172.21.15.38:0/4280160809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb03e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- 172.21.15.38:0/4280160809 >> 172.21.15.38:0/4280160809 conn(0x55a7b1e9d070 msgr2=0x55a7b1e980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- 172.21.15.38:0/4280160809 shutdown_connections 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- 172.21.15.38:0/4280160809 wait complete. 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- start start 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 Processor -- start 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1ebe780 con 0x55a7b1ea5f90 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1ebe0b0 con 0x55a7b1ea7b20 2024-04-25T13:04:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1ebee50 con 0x55a7b1ea7160 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.529+0000 7f8a669a7740 10 client.0 _ll_register_callbacks cb 0x7f8a5800b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47632/0 (socket says 172.21.15.38:47632) 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 -- 172.21.15.38:0/3955153393 learned_addr learned my addr 172.21.15.38:0/3955153393 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61949640 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 -- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 msgr2=0x55a7b1eb4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 -- 172.21.15.38:0/3955153393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 msgr2=0x55a7b1eb6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 -- 172.21.15.38:0/3955153393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b1eb6f20 con 0x55a7b1ea5f90 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a61148640 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb3b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c010010 tx=0x7f8a4c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36e371ac4128aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a457fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a457fa640 1 -- 172.21.15.38:0/3955153393 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c031070 con 0x55a7b1ea5f90 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a457fa640 1 -- 172.21.15.38:0/3955153393 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8a4c005250 con 0x55a7b1ea5f90 2024-04-25T13:04:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c00fc60 con 0x55a7b1ea5f90 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a457fa640 1 -- 172.21.15.38:0/3955153393 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c005c90 con 0x55a7b1ea5f90 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a669a7740 10 client.15267 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a7b1f23770 con 0x55a7b1ea5f90 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.530+0000 7f8a457fa640 1 -- 172.21.15.38:0/3955153393 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8a4c00fc60 con 0x55a7b1ea5f90 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a457fa640 1 -- 172.21.15.38:0/3955153393 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a4c035070 con 0x55a7b1ea5f90 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a669a7740 10 client.15267 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a669a7740 10 client.15267 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a669a7740 20 client.15267 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1f27b00 0x55a7b1f26810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a669a7740 4 client.15267 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 --> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] -- command(tid 0: {"prefix": "status"}) -- 0x55a7b1e0e4e0 con 0x55a7b1f27b00 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a61949640 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1f27b00 0x55a7b1f26810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a61949640 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1f27b00 0x55a7b1f26810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3db7c3035fe9ec1e server_cookie=106ecfb57453442 in_seq=0 out_seq=0 2024-04-25T13:04:39.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.531+0000 7f8a457fa640 10 client.15267 ms_handle_connect on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a457fa640 1 -- 172.21.15.38:0/3955153393 <== mds.3 v2:172.21.15.178:6836/2053792208 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a7b1e0e4e0 con 0x55a7b1f27b00 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a457fa640 10 client.15267 handle_command_reply: tid=0 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 client.15267 shutdown 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 msgr2=0x55a7b1eb3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb3b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c010010 tx=0x7f8a4c005050 comp rx=0 tx=0).stop 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 shutdown_connections 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:6836/2053792208,v1:172.21.15.178:6837/2053792208] conn(0x55a7b1f27b00 0x55a7b1f26810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a7b1ea7b20 0x55a7b1eb6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7b1ea7160 0x55a7b1eb4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 --2- 172.21.15.38:0/3955153393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a7b1ea5f90 0x55a7b1eb3b50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a457fa640 0 client.15267 ms_handle_reset on v2:172.21.15.178:6836/2053792208 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 >> 172.21.15.38:0/3955153393 conn(0x55a7b1e9d070 msgr2=0x55a7b1eb03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 shutdown_connections 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 1 -- 172.21.15.38:0/3955153393 wait complete. 2024-04-25T13:04:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a669a7740 20 client.15267 trim_cache size 0 max 16384 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24466, 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 109.57245210000001, 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:39.538 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 133.219488813 2024-04-25T13:04:39.539 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8a480a3bb0 msgr2=0x7f8a480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8a480a3bb0 0x7f8a480a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a540ba1f0 tx=0x7f8a540de040 comp rx=0 tx=0).stop 2024-04-25T13:04:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 msgr2=0x7f8a5c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c19f800 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009dc0 tx=0x7f8a54031840 comp rx=0 tx=0).stop 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 shutdown_connections 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f8a480a3bb0 0x7f8a480a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a5c107cf0 0x7f8a5c19f800 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c1053c0 0x7f8a5c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 --2- 172.21.15.38:0/2588463864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a5c102a90 0x7f8a5c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.532+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 >> 172.21.15.38:0/2588463864 conn(0x7f8a5c0fc440 msgr2=0x7f8a5c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.533+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 shutdown_connections 2024-04-25T13:04:39.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.533+0000 7f8a63bad640 1 -- 172.21.15.38:0/2588463864 wait complete. 2024-04-25T13:04:39.541 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:39 smithi038 ceph-mon[29400]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:39.541 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:39 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1439091496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:39.541 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:39 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2362784298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:39.541 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:39 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/187223911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:39.542 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T13:04:39.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:39 smithi104 ceph-mon[32726]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T13:04:39.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:39 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1439091496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:04:39.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:39 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2362784298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:04:39.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:39 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/187223911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 -- 172.21.15.38:0/1530598552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570102a30 msgr2=0x7ff570104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 --2- 172.21.15.38:0/1530598552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570102a30 0x7ff570104e20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff560009ca0 tx=0x7ff56002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 -- 172.21.15.38:0/1530598552 shutdown_connections 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 --2- 172.21.15.38:0/1530598552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff570107c90 0x7ff57010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 --2- 172.21.15.38:0/1530598552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570105360 0x7ff570107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 --2- 172.21.15.38:0/1530598552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570102a30 0x7ff570104e20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 -- 172.21.15.38:0/1530598552 >> 172.21.15.38:0/1530598552 conn(0x7ff5700fc440 msgr2=0x7ff5700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 -- 172.21.15.38:0/1530598552 shutdown_connections 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.646+0000 7ff5775ed640 1 -- 172.21.15.38:0/1530598552 wait complete. 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 Processor -- start 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 -- start start 2024-04-25T13:04:39.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff570102a30 0x7ff57019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 0x7ff57019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570107c90 0x7ff5701a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff57010dc30 con 0x7ff570105360 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff57010dab0 con 0x7ff570102a30 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff5775ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff57010ddb0 con 0x7ff570107c90 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 0x7ff57019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 0x7ff57019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47654/0 (socket says 172.21.15.38:47654) 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 -- 172.21.15.38:0/980834833 learned_addr learned my addr 172.21.15.38:0/980834833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff575b8a640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570107c90 0x7ff5701a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 -- 172.21.15.38:0/980834833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570107c90 msgr2=0x7ff5701a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff575389640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff570102a30 0x7ff57019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570107c90 0x7ff5701a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 -- 172.21.15.38:0/980834833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff570102a30 msgr2=0x7ff57019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff570102a30 0x7ff57019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 -- 172.21.15.38:0/980834833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5701a2100 con 0x7ff570105360 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff575b8a640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570107c90 0x7ff5701a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff574b88640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 0x7ff57019aa20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c970 tx=0x7ff56400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.647+0000 7ff56e7fc640 1 -- 172.21.15.38:0/980834833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564007bb0 con 0x7ff570105360 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.648+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5701a23f0 con 0x7ff570105360 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.648+0000 7ff56e7fc640 1 -- 172.21.15.38:0/980834833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff564007d50 con 0x7ff570105360 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.648+0000 7ff56e7fc640 1 -- 172.21.15.38:0/980834833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564018780 con 0x7ff570105360 2024-04-25T13:04:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.648+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5701a2940 con 0x7ff570105360 2024-04-25T13:04:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff56e7fc640 1 -- 172.21.15.38:0/980834833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff56400fa40 con 0x7ff570105360 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 Processor -- start 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 -- start start 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750268fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 0x55e75026a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e75026a710 0x55e750270fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e7501cdc50 con 0x55e750269d50 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e75016a830 con 0x55e750268b80 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e7501c9c10 con 0x55e75026a710 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff56e7fc640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff5580a3bb0 0x7ff5580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff575389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 0x55e75026a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff575389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 0x55e75026a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47660/0 (socket says 172.21.15.38:47660) 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff575b8a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750268fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.650+0000 7ff575389640 1 -- 172.21.15.38:0/1507378807 learned_addr learned my addr 172.21.15.38:0/1507378807 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575b8a640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff5580a3bb0 0x7ff5580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff56e7fc640 1 -- 172.21.15.38:0/980834833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff5640d05b0 con 0x7ff570105360 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff574b88640 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e75026a710 0x55e750270fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575389640 1 -- 172.21.15.38:0/1507378807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e75026a710 msgr2=0x55e750270fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575389640 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e75026a710 0x55e750270fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575389640 1 -- 172.21.15.38:0/1507378807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 msgr2=0x55e750268fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575389640 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750268fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575389640 1 -- 172.21.15.38:0/1507378807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74fee02d0 con 0x55e750269d50 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750268fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575389640 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 0x55e75026a1d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff560000f80 tx=0x7ff560009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bce8b247ae9ad4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff575b8a640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff5580a3bb0 0x7ff5580a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55e750269c50 tx=0x7ff568005b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff5577fe640 1 -- 172.21.15.38:0/1507378807 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560031790 con 0x55e750269d50 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff5577fe640 1 -- 172.21.15.38:0/1507378807 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff560036070 con 0x55e750269d50 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.651+0000 7ff5577fe640 1 -- 172.21.15.38:0/1507378807 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56002f660 con 0x55e750269d50 2024-04-25T13:04:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1507378807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 msgr2=0x55e75026a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 0x55e75026a1d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff560000f80 tx=0x7ff560009880 comp rx=0 tx=0).stop 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1507378807 shutdown_connections 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e75026a710 0x55e750270fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750269d50 0x55e75026a1d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1507378807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750268fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1507378807 >> 172.21.15.38:0/1507378807 conn(0x55e750264070 msgr2=0x55e750264470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1507378807 shutdown_connections 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1507378807 wait complete. 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 -- start start 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.652+0000 7ff57a3d8740 1 Processor -- start 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e7502716c0 con 0x55e75026a710 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e750168470 con 0x55e750268b80 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e750271840 con 0x55e750269d50 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff57a3d8740 10 client.0 _ll_register_callbacks cb 0x55e75018df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47676/0 (socket says 172.21.15.38:47676) 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 -- 172.21.15.38:0/1335711567 learned_addr learned my addr 172.21.15.38:0/1335711567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff574b88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575389640 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 -- 172.21.15.38:0/1335711567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 msgr2=0x55e750273490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 -- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 msgr2=0x55e750272f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 -- 172.21.15.38:0/1335711567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74ffaa6b0 con 0x55e75026a710 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a4d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff56800fa80 tx=0x7ff5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363944ecfad3ee6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.653+0000 7ff554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff554ff9640 1 -- 172.21.15.38:0/1335711567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56801cd00 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff554ff9640 1 -- 172.21.15.38:0/1335711567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff56801e450 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff560036070 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff554ff9640 1 -- 172.21.15.38:0/1335711567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff56801e650 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e75027d8b0 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff554ff9640 1 -- 172.21.15.38:0/1335711567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff560036070 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff554ff9640 1 -- 172.21.15.38:0/1335711567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff56801b040 con 0x55e75026a710 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 10 client.15279 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 20 client.15279 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e75027ff60 0x55e7502813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 4 client.15279 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.654+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 --> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7501db670 con 0x55e75027ff60 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.655+0000 7ff575389640 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e75027ff60 0x55e7502813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.655+0000 7ff575389640 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e75027ff60 0x55e7502813d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3de1bc32e68f4284 server_cookie=19a9715e9ffebe70 in_seq=0 out_seq=0 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.655+0000 7ff554ff9640 10 client.15279 ms_handle_connect on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff554ff9640 1 -- 172.21.15.38:0/1335711567 <== mds.2 v2:172.21.15.104:6836/2247876378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7501db670 con 0x55e75027ff60 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff554ff9640 10 client.15279 handle_command_reply: tid=0 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 client.15279 shutdown 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 msgr2=0x55e75027a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a4d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff56800fa80 tx=0x7ff5680046c0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 shutdown_connections 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e75027ff60 0x55e7502813d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a4d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1335711567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff554ff9640 0 client.15279 ms_handle_reset on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 >> 172.21.15.38:0/1335711567 conn(0x55e750264070 msgr2=0x55e7502631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 shutdown_connections 2024-04-25T13:04:39.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1335711567 wait complete. 2024-04-25T13:04:39.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.656+0000 7ff57a3d8740 20 client.15279 trim_cache size 0 max 16384 2024-04-25T13:04:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 Processor -- start 2024-04-25T13:04:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 -- start start 2024-04-25T13:04:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e75027f710 con 0x55e75026a710 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e75027fde0 con 0x55e750269d50 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e7502804b0 con 0x55e750268b80 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff575b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff575b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47692/0 (socket says 172.21.15.38:47692) 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.674+0000 7ff575b8a640 1 -- 172.21.15.38:0/1438019405 learned_addr learned my addr 172.21.15.38:0/1438019405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff574b88640 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff575b8a640 1 -- 172.21.15.38:0/1438019405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750268b80 msgr2=0x55e750272f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff575b8a640 1 -- 172.21.15.38:0/1438019405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750269d50 msgr2=0x55e750273490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff575b8a640 1 -- 172.21.15.38:0/1438019405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7502697d0 con 0x55e75026a710 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff575b8a640 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d5a0 tx=0x7ff56800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dd39a40e74301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff556ffd640 1 -- 172.21.15.38:0/1438019405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56801c0b0 con 0x55e75026a710 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff556ffd640 1 -- 172.21.15.38:0/1438019405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff568032400 con 0x55e75026a710 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.675+0000 7ff556ffd640 1 -- 172.21.15.38:0/1438019405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56801ebe0 con 0x55e75026a710 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1438019405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 msgr2=0x55e75027a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d5a0 tx=0x7ff56800df40 comp rx=0 tx=0).stop 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1438019405 shutdown_connections 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e75026a710 0x55e75027a3f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e750269d50 0x55e750273490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/1438019405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750268b80 0x55e750272f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1438019405 >> 172.21.15.38:0/1438019405 conn(0x55e750264070 msgr2=0x55e75025ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1438019405 shutdown_connections 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- 172.21.15.38:0/1438019405 wait complete. 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- start start 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 Processor -- start 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 0x55e750275bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750276100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e75026a710 0x55e750279120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e750281250 con 0x55e750268b80 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e750280b80 con 0x55e75026a710 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e750281920 con 0x55e750269d50 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff57a3d8740 10 client.0 _ll_register_callbacks cb 0x7ff568005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff575389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 0x55e750275bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.676+0000 7ff575389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 0x55e750275bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44264/0 (socket says 172.21.15.38:44264) 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 -- 172.21.15.38:0/604906969 learned_addr learned my addr 172.21.15.38:0/604906969 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575b8a640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e75026a710 0x55e750279120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff574b88640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750276100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 -- 172.21.15.38:0/604906969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 msgr2=0x55e750276100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750276100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 -- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e75026a710 msgr2=0x55e750279120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e75026a710 0x55e750279120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 -- 172.21.15.38:0/604906969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e750279710 con 0x55e750268b80 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff574b88640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750276100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff575389640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 0x55e750275bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff560000f80 tx=0x7ff56002f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39640223c401c893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff554ff9640 1 -- 172.21.15.38:0/604906969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560049040 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff554ff9640 1 -- 172.21.15.38:0/604906969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff560031790 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff568032400 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff554ff9640 1 -- 172.21.15.38:0/604906969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5600511b0 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff57a3d8740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7502e5f10 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.677+0000 7ff554ff9640 1 -- 172.21.15.38:0/604906969 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff568032400 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff554ff9640 1 -- 172.21.15.38:0/604906969 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff56002fb20 con 0x55e750268b80 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff57a3d8740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff57a3d8740 10 client.15285 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff57a3d8740 20 client.15285 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e7502ea610 0x55e7502e9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff57a3d8740 4 client.15285 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 --> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] -- command(tid 0: {"prefix": "status"}) -- 0x55e7501d54e0 con 0x55e7502ea610 2024-04-25T13:04:39.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff574b88640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e7502ea610 0x55e7502e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff554ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.678+0000 7ff574b88640 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e7502ea610 0x55e7502e9320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b17466c9dc5347c server_cookie=4775abe05ef89ea5 in_seq=0 out_seq=0 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff554ff9640 1 -- 172.21.15.38:0/604906969 <== mds.2 v2:172.21.15.104:6836/2247876378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e7501d54e0 con 0x55e7502ea610 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff554ff9640 10 client.15285 handle_command_reply: tid=0 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 client.15285 shutdown 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 msgr2=0x55e750275bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 0x55e750275bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff560000f80 tx=0x7ff56002f530 comp rx=0 tx=0).stop 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 shutdown_connections 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:6836/2247876378,v1:172.21.15.104:6837/2247876378] conn(0x55e7502ea610 0x55e7502e9320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e75026a710 0x55e750279120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e750269d50 0x55e750276100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 --2- 172.21.15.38:0/604906969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e750268b80 0x55e750275bc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff554ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.104:6836/2247876378 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 >> 172.21.15.38:0/604906969 conn(0x55e750264070 msgr2=0x55e750272f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 shutdown_connections 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 1 -- 172.21.15.38:0/604906969 wait complete. 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff57a3d8740 20 client.15285 trim_cache size 0 max 16384 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24482, 2024-04-25T13:04:39.686 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 110.738774761, 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 130.424972045 2024-04-25T13:04:39.687 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff5580a3bb0 msgr2=0x7ff5580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.679+0000 7ff5775ed640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff5580a3bb0 0x7ff5580a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55e750269c50 tx=0x7ff568005b00 comp rx=0 tx=0).stop 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 msgr2=0x7ff57019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 0x7ff57019aa20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c970 tx=0x7ff56400ce30 comp rx=0 tx=0).stop 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 shutdown_connections 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7ff5580a3bb0 0x7ff5580a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570107c90 0x7ff5701a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff570105360 0x7ff57019aa20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 --2- 172.21.15.38:0/980834833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff570102a30 0x7ff57019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 >> 172.21.15.38:0/980834833 conn(0x7ff5700fc440 msgr2=0x7ff570102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 shutdown_connections 2024-04-25T13:04:39.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.680+0000 7ff5775ed640 1 -- 172.21.15.38:0/980834833 wait complete. 2024-04-25T13:04:39.690 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:04:39.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 -- 172.21.15.38:0/4090182225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103b50 msgr2=0x7f40f010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/4090182225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103b50 0x7f40f010a3e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009900 tx=0x7f40d802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:39.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 -- 172.21.15.38:0/4090182225 shutdown_connections 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/4090182225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103b50 0x7f40f010a3e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/4090182225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f0103190 0x7f40f0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/4090182225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f0101fa0 0x7f40f01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 -- 172.21.15.38:0/4090182225 >> 172.21.15.38:0/4090182225 conn(0x7f40f00fd530 msgr2=0x7f40f00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 -- 172.21.15.38:0/4090182225 shutdown_connections 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.794+0000 7f40f6bb4640 1 -- 172.21.15.38:0/4090182225 wait complete. 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 Processor -- start 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 -- start start 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f0101fa0 0x7f40f0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 0x7f40f01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f0103b50 0x7f40f019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010dc50 con 0x7f40f0103190 2024-04-25T13:04:39.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010dad0 con 0x7f40f0103b50 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f6bb4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010ddd0 con 0x7f40f0101fa0 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 0x7f40f01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 0x7f40f01987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44278/0 (socket says 172.21.15.38:44278) 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40effff640 1 -- 172.21.15.38:0/2008115401 learned_addr learned my addr 172.21.15.38:0/2008115401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f4950640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f0101fa0 0x7f40f0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.795+0000 7f40f5151640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f0103b50 0x7f40f019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40effff640 1 -- 172.21.15.38:0/2008115401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f0101fa0 msgr2=0x7f40f0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40effff640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f0101fa0 0x7f40f0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40effff640 1 -- 172.21.15.38:0/2008115401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f0103b50 msgr2=0x7f40f019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40effff640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f0103b50 0x7f40f019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40effff640 1 -- 172.21.15.38:0/2008115401 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f019fe80 con 0x7f40f0103190 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40effff640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 0x7f40f01987a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c970 tx=0x7f40e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40edffb640 1 -- 172.21.15.38:0/2008115401 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007bb0 con 0x7f40f0103190 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40edffb640 1 -- 172.21.15.38:0/2008115401 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f40e0007d50 con 0x7f40f0103190 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f01a0170 con 0x7f40f0103190 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40edffb640 1 -- 172.21.15.38:0/2008115401 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e000f450 con 0x7f40f0103190 2024-04-25T13:04:39.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.796+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f01a06c0 con 0x7f40f0103190 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.798+0000 7f40edffb640 1 -- 172.21.15.38:0/2008115401 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f40e00046c0 con 0x7f40f0103190 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.798+0000 7f40f99a9740 1 Processor -- start 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 -- start start 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40edffb640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f40dc0a3c30 0x7f40dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc7fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc7fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc803fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ebc760c50 con 0x557ebc7fd710 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ebc6fd830 con 0x557ebc7fbb80 2024-04-25T13:04:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f99a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557ebc75cc10 con 0x557ebc7fcd50 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f5151640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f40dc0a3c30 0x7f40dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc803fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc803fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44294/0 (socket says 172.21.15.38:44294) 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f5151640 1 -- 172.21.15.38:0/877089391 learned_addr learned my addr 172.21.15.38:0/877089391 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f4950640 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc7fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40edffb640 1 -- 172.21.15.38:0/2008115401 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f40e00cffc0 con 0x7f40f0103190 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40effff640 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc7fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f4950640 1 -- 172.21.15.38:0/877089391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 msgr2=0x557ebc7fd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f4950640 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc7fd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f4950640 1 -- 172.21.15.38:0/877089391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 msgr2=0x557ebc803fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f4950640 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc803fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f4950640 1 -- 172.21.15.38:0/877089391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ebc4732d0 con 0x557ebc7fbb80 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f5151640 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc803fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:39.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.799+0000 7f40f5151640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f40dc0a3c30 0x7f40dc0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40d80096c0 tx=0x7f40d80040d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40f4950640 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc7fbfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005fd0 tx=0x7f40e8003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290ced007b04710d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40e6ffd640 1 -- 172.21.15.38:0/877089391 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004b20 con 0x557ebc7fbb80 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40e6ffd640 1 -- 172.21.15.38:0/877089391 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f40e8004cc0 con 0x557ebc7fbb80 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40e6ffd640 1 -- 172.21.15.38:0/877089391 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80126b0 con 0x557ebc7fbb80 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40f99a9740 1 -- 172.21.15.38:0/877089391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 msgr2=0x557ebc7fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40f99a9740 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc7fbfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005fd0 tx=0x7f40e8003e80 comp rx=0 tx=0).stop 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.800+0000 7f40f99a9740 1 -- 172.21.15.38:0/877089391 shutdown_connections 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc803fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc7fd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 --2- 172.21.15.38:0/877089391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc7fbfa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- 172.21.15.38:0/877089391 >> 172.21.15.38:0/877089391 conn(0x557ebc7f7070 msgr2=0x557ebc7f7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- 172.21.15.38:0/877089391 shutdown_connections 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- 172.21.15.38:0/877089391 wait complete. 2024-04-25T13:04:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- start start 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 Processor -- start 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc805fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc8064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc80d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ebc804720 con 0x557ebc7fd710 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ebc6fb470 con 0x557ebc7fbb80 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557ebc8048a0 con 0x557ebc7fcd50 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f99a9740 10 client.0 _ll_register_callbacks cb 0x557ebc720f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc80d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc80d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44304/0 (socket says 172.21.15.38:44304) 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f5151640 1 -- 172.21.15.38:0/2319180508 learned_addr learned my addr 172.21.15.38:0/2319180508 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.801+0000 7f40f4950640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc805fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40effff640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc8064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f5151640 1 -- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 msgr2=0x557ebc8064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f5151640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc8064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f5151640 1 -- 172.21.15.38:0/2319180508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 msgr2=0x557ebc805fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f5151640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc805fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f5151640 1 -- 172.21.15.38:0/2319180508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ebc53d6b0 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40effff640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc8064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f5151640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc80d520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f40d8005780 tx=0x7f40d8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6f6f9f3c565b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40cffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40cffff640 1 -- 172.21.15.38:0/2319180508 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d804f030 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40cffff640 1 -- 172.21.15.38:0/2319180508 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f40d8032250 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e8004cc0 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40cffff640 1 -- 172.21.15.38:0/2319180508 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40d8032450 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f99a9740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ebc810900 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.802+0000 7f40cffff640 1 -- 172.21.15.38:0/2319180508 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f40e8004cc0 con 0x557ebc7fd710 2024-04-25T13:04:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40cffff640 1 -- 172.21.15.38:0/2319180508 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40d80038f0 con 0x557ebc7fd710 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f99a9740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f99a9740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f99a9740 20 client.15297 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f99a9740 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc812fb0 0x557ebc814420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f99a9740 4 client.15297 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 --> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ebc76e670 con 0x557ebc812fb0 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f4950640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc812fb0 0x557ebc814420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40cffff640 10 client.15297 ms_handle_connect on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.803+0000 7f40f4950640 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc812fb0 0x557ebc814420 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4f276c320df8849 server_cookie=4df809a6f6a495d9 in_seq=0 out_seq=0 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40cffff640 1 -- 172.21.15.38:0/2319180508 <== mds.1 v2:172.21.15.178:6838/391098246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ebc76e670 con 0x557ebc812fb0 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40cffff640 10 client.15297 handle_command_reply: tid=0 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 client.15297 shutdown 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 msgr2=0x557ebc80d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc80d520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f40d8005780 tx=0x7f40d8005a20 comp rx=0 tx=0).stop 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 shutdown_connections 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc80d520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc8064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc812fb0 0x557ebc814420 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 --2- 172.21.15.38:0/2319180508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc805fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 >> 172.21.15.38:0/2319180508 conn(0x557ebc7f7070 msgr2=0x557ebc7f61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40cffff640 0 client.15297 ms_handle_reset on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 shutdown_connections 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 1 -- 172.21.15.38:0/2319180508 wait complete. 2024-04-25T13:04:39.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.804+0000 7f40f99a9740 20 client.15297 trim_cache size 0 max 16384 2024-04-25T13:04:39.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 Processor -- start 2024-04-25T13:04:39.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 -- start start 2024-04-25T13:04:39.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fbb80 0x557ebc811d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fcd50 0x557ebc80d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc805fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ebc812760 con 0x557ebc7fd710 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ebc812e30 con 0x557ebc7fcd50 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f99a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557ebc813500 con 0x557ebc7fbb80 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc805fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc805fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44320/0 (socket says 172.21.15.38:44320) 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 -- 172.21.15.38:0/901358196 learned_addr learned my addr 172.21.15.38:0/901358196 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f4950640 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fbb80 0x557ebc811d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40effff640 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fcd50 0x557ebc80d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 -- 172.21.15.38:0/901358196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fbb80 msgr2=0x557ebc811d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fbb80 0x557ebc811d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 -- 172.21.15.38:0/901358196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fcd50 msgr2=0x557ebc80d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fcd50 0x557ebc80d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 -- 172.21.15.38:0/901358196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ebc7fc7d0 con 0x557ebc7fd710 2024-04-25T13:04:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.823+0000 7f40f5151640 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc805fa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40d802a930 tx=0x7f40d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639b34cbb6e313bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40e67fc640 1 -- 172.21.15.38:0/901358196 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d804f040 con 0x557ebc7fd710 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40e67fc640 1 -- 172.21.15.38:0/901358196 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f40d8031610 con 0x557ebc7fd710 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40e67fc640 1 -- 172.21.15.38:0/901358196 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d804c6d0 con 0x557ebc7fd710 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 -- 172.21.15.38:0/901358196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 msgr2=0x557ebc805fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc805fa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40d802a930 tx=0x7f40d8005fd0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 -- 172.21.15.38:0/901358196 shutdown_connections 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc805fa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fcd50 0x557ebc80d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 --2- 172.21.15.38:0/901358196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fbb80 0x557ebc811d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 -- 172.21.15.38:0/901358196 >> 172.21.15.38:0/901358196 conn(0x557ebc7f7070 msgr2=0x557ebc7f20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 -- 172.21.15.38:0/901358196 shutdown_connections 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.824+0000 7f40f99a9740 1 -- 172.21.15.38:0/901358196 wait complete. 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 -- start start 2024-04-25T13:04:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 Processor -- start 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc8084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc808a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc876f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ebc8142a0 con 0x557ebc7fd710 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ebc813bd0 con 0x557ebc7fbb80 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557ebc814970 con 0x557ebc7fcd50 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f99a9740 10 client.0 _ll_register_callbacks cb 0x7f40d8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc876f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc876f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44322/0 (socket says 172.21.15.38:44322) 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 -- 172.21.15.38:0/1696630989 learned_addr learned my addr 172.21.15.38:0/1696630989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f4950640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc8084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40effff640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc808a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 -- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 msgr2=0x557ebc808a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc808a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 -- 172.21.15.38:0/1696630989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 msgr2=0x557ebc8084c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc8084c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.825+0000 7f40f5151640 1 -- 172.21.15.38:0/1696630989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ebc877510 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f5151640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc876f50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f40d802aa00 tx=0x7f40d8003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dbf600aa78fefc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40cffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40cffff640 1 -- 172.21.15.38:0/1696630989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8063070 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40cffff640 1 -- 172.21.15.38:0/1696630989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f40d80319d0 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d8031610 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40cffff640 1 -- 172.21.15.38:0/1696630989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40d804f040 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ebc878810 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40cffff640 1 -- 172.21.15.38:0/1696630989 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f40d8031610 con 0x557ebc7fd710 2024-04-25T13:04:39.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40cffff640 1 -- 172.21.15.38:0/1696630989 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40d805f460 con 0x557ebc7fd710 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 10 client.15303 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 20 client.15303 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc87d020 0x557ebc879b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 4 client.15303 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.826+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 --> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] -- command(tid 0: {"prefix": "status"}) -- 0x557ebc7684e0 con 0x557ebc87d020 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f4950640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc87d020 0x557ebc879b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40cffff640 10 client.15303 ms_handle_connect on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f4950640 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc87d020 0x557ebc879b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fc34911dcd92080 server_cookie=76a56758efc6e6 in_seq=0 out_seq=0 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40cffff640 1 -- 172.21.15.38:0/1696630989 <== mds.1 v2:172.21.15.178:6838/391098246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x557ebc7684e0 con 0x557ebc87d020 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40cffff640 10 client.15303 handle_command_reply: tid=0 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 client.15303 shutdown 2024-04-25T13:04:39.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 msgr2=0x557ebc876f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc876f50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f40d802aa00 tx=0x7f40d8003af0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 shutdown_connections 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ebc7fd710 0x557ebc876f50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ebc7fcd50 0x557ebc808a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.178:6838/391098246,v1:172.21.15.178:6839/391098246] conn(0x557ebc87d020 0x557ebc879b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 --2- 172.21.15.38:0/1696630989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ebc7fbb80 0x557ebc8084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40cffff640 0 client.15303 ms_handle_reset on v2:172.21.15.178:6838/391098246 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.827+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 >> 172.21.15.38:0/1696630989 conn(0x557ebc7f7070 msgr2=0x557ebc811d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 shutdown_connections 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f99a9740 1 -- 172.21.15.38:0/1696630989 wait complete. 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f99a9740 20 client.15303 trim_cache size 0 max 16384 2024-04-25T13:04:39.833 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24484, 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 111.89870209, 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 127.744035815 2024-04-25T13:04:39.834 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:39.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f40dc0a3c30 msgr2=0x7f40dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f40dc0a3c30 0x7f40dc0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40d80096c0 tx=0x7f40d80040d0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 msgr2=0x7f40f01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 0x7f40f01987a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c970 tx=0x7f40e000ce30 comp rx=0 tx=0).stop 2024-04-25T13:04:39.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 shutdown_connections 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f40dc0a3c30 0x7f40dc0a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f0103b50 0x7f40f019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40f0103190 0x7f40f01987a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 --2- 172.21.15.38:0/2008115401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f0101fa0 0x7f40f0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 >> 172.21.15.38:0/2008115401 conn(0x7f40f00fd530 msgr2=0x7f40f0106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 shutdown_connections 2024-04-25T13:04:39.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.828+0000 7f40f6bb4640 1 -- 172.21.15.38:0/2008115401 wait complete. 2024-04-25T13:04:39.838 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 -- 172.21.15.38:0/1000639226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0107c90 msgr2=0x7f23e010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/1000639226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0107c90 0x7f23e010a0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009ca0 tx=0x7f23d802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 -- 172.21.15.38:0/1000639226 shutdown_connections 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/1000639226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0107c90 0x7f23e010a0e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/1000639226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e0105360 0x7f23e0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/1000639226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0102a30 0x7f23e0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 -- 172.21.15.38:0/1000639226 >> 172.21.15.38:0/1000639226 conn(0x7f23e00fc440 msgr2=0x7f23e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 -- 172.21.15.38:0/1000639226 shutdown_connections 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 -- 172.21.15.38:0/1000639226 wait complete. 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.942+0000 7f23e5c2a640 1 Processor -- start 2024-04-25T13:04:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 -- start start 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e0102a30 0x7f23e01023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 0x7f23e01009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0107c90 0x7f23e0100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23e010dc90 con 0x7f23e0105360 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23e010db10 con 0x7f23e0102a30 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23e5c2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23e010de10 con 0x7f23e0107c90 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 0x7f23e01009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 0x7f23e01009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44344/0 (socket says 172.21.15.38:44344) 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 -- 172.21.15.38:0/2162759450 learned_addr learned my addr 172.21.15.38:0/2162759450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23dffff640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0107c90 0x7f23e0100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 -- 172.21.15.38:0/2162759450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0107c90 msgr2=0x7f23e0100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0107c90 0x7f23e0100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23df7fe640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e0102a30 0x7f23e01023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 -- 172.21.15.38:0/2162759450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e0102a30 msgr2=0x7f23e01023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e0102a30 0x7f23e01023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 -- 172.21.15.38:0/2162759450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e01017b0 con 0x7f23e0105360 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23dffff640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0107c90 0x7f23e0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.943+0000 7f23deffd640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 0x7f23e01009e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c970 tx=0x7f23d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.944+0000 7f23dcff9640 1 -- 172.21.15.38:0/2162759450 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4014070 con 0x7f23e0105360 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.944+0000 7f23dcff9640 1 -- 172.21.15.38:0/2162759450 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23d4004440 con 0x7f23e0105360 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.944+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e01a2340 con 0x7f23e0105360 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.944+0000 7f23dcff9640 1 -- 172.21.15.38:0/2162759450 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4003e40 con 0x7f23e0105360 2024-04-25T13:04:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.944+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e01a2910 con 0x7f23e0105360 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23dcff9640 1 -- 172.21.15.38:0/2162759450 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f23d4017070 con 0x7f23e0105360 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 Processor -- start 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 -- start start 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d879fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d87b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d881fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56439d7dec50 con 0x56439d879b80 2024-04-25T13:04:39.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56439d77b830 con 0x56439d87b710 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56439d7dac10 con 0x56439d87ad50 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d879fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d879fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44350/0 (socket says 172.21.15.38:44350) 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23dffff640 1 -- 172.21.15.38:0/851655117 learned_addr learned my addr 172.21.15.38:0/851655117 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23deffd640 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d881fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.946+0000 7f23dcff9640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f23c80a3bb0 0x7f23c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23df7fe640 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d87b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 -- 172.21.15.38:0/851655117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 msgr2=0x56439d87b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d87b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 -- 172.21.15.38:0/851655117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 msgr2=0x56439d881fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dcff9640 1 -- 172.21.15.38:0/2162759450 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f23d40d0320 con 0x7f23e0105360 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d881fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 -- 172.21.15.38:0/851655117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439d4f12d0 con 0x56439d879b80 2024-04-25T13:04:39.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23deffd640 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d881fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f23c80a3bb0 0x7f23c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23dffff640 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d879fa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009a00 tx=0x7f23d8031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9546d87a690e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23c5ffb640 1 -- 172.21.15.38:0/851655117 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004510 con 0x56439d879b80 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23c5ffb640 1 -- 172.21.15.38:0/851655117 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23d8036070 con 0x56439d879b80 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.947+0000 7f23c5ffb640 1 -- 172.21.15.38:0/851655117 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d802fed0 con 0x56439d879b80 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 -- 172.21.15.38:0/851655117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 msgr2=0x56439d879fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23dffff640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f23c80a3bb0 0x7f23c80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23d804b000 tx=0x7f23d8048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d879fa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009a00 tx=0x7f23d8031450 comp rx=0 tx=0).stop 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 -- 172.21.15.38:0/851655117 shutdown_connections 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d881fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d87b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/851655117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d879fa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 -- 172.21.15.38:0/851655117 >> 172.21.15.38:0/851655117 conn(0x56439d875070 msgr2=0x56439d875470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 -- 172.21.15.38:0/851655117 shutdown_connections 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 -- 172.21.15.38:0/851655117 wait complete. 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.948+0000 7f23e8a1a740 1 -- start start 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 Processor -- start 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d883e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d8843c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d88b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56439d8825a0 con 0x56439d879b80 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56439d779470 con 0x56439d87b710 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56439d882720 con 0x56439d87ad50 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 10 client.0 _ll_register_callbacks cb 0x56439d79ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d883e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d883e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44358/0 (socket says 172.21.15.38:44358) 2024-04-25T13:04:39.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 -- 172.21.15.38:0/1649643642 learned_addr learned my addr 172.21.15.38:0/1649643642 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23deffd640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d8843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23dffff640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d88b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 -- 172.21.15.38:0/1649643642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 msgr2=0x56439d8843c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d8843c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 -- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 msgr2=0x56439d88b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d88b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 -- 172.21.15.38:0/1649643642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439d5bb6b0 con 0x56439d879b80 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23dffff640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d88b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23df7fe640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d883e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f23d000e280 tx=0x7f23d000e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b15421d7507441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23beffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23beffd640 1 -- 172.21.15.38:0/1649643642 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0018020 con 0x56439d879b80 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23beffd640 1 -- 172.21.15.38:0/1649643642 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f23d0013660 con 0x56439d879b80 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.949+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d8036070 con 0x56439d879b80 2024-04-25T13:04:39.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23beffd640 1 -- 172.21.15.38:0/1649643642 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23d0013860 con 0x56439d879b80 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56439d88e840 con 0x56439d879b80 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23beffd640 1 -- 172.21.15.38:0/1649643642 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23d8036070 con 0x56439d879b80 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23beffd640 1 -- 172.21.15.38:0/1649643642 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23d0014be0 con 0x56439d879b80 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 10 client.15315 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 20 client.15315 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d890ea0 0x56439d892310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 4 client.15315 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 --> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56439d7ec670 con 0x56439d890ea0 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.950+0000 7f23deffd640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d890ea0 0x56439d892310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.951+0000 7f23deffd640 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d890ea0 0x56439d892310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d6d6498dc45054 server_cookie=ba86e4f4f22a5a57 in_seq=0 out_seq=0 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.951+0000 7f23beffd640 10 client.15315 ms_handle_connect on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:39.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.951+0000 7f23beffd640 1 -- 172.21.15.38:0/1649643642 <== mds.0 v2:172.21.15.104:6838/1961595450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56439d7ec670 con 0x56439d890ea0 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23beffd640 10 client.15315 handle_command_reply: tid=0 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 client.15315 shutdown 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 msgr2=0x56439d883e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d883e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f23d000e280 tx=0x7f23d000e740 comp rx=0 tx=0).stop 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 shutdown_connections 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d88b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d890ea0 0x56439d892310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d8843c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/1649643642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d883e80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23beffd640 0 client.15315 ms_handle_reset on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 >> 172.21.15.38:0/1649643642 conn(0x56439d875070 msgr2=0x56439d8741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 shutdown_connections 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 1 -- 172.21.15.38:0/1649643642 wait complete. 2024-04-25T13:04:39.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.952+0000 7f23e8a1a740 20 client.15315 trim_cache size 0 max 16384 2024-04-25T13:04:39.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 Processor -- start 2024-04-25T13:04:39.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 -- start start 2024-04-25T13:04:39.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d881e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d88b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d8823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56439d890650 con 0x56439d879b80 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56439d890d20 con 0x56439d87b710 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56439d8913f0 con 0x56439d87ad50 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d881e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d881e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44366/0 (socket says 172.21.15.38:44366) 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 -- 172.21.15.38:0/2941907693 learned_addr learned my addr 172.21.15.38:0/2941907693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 -- 172.21.15.38:0/2941907693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 msgr2=0x56439d88b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d88b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 -- 172.21.15.38:0/2941907693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 msgr2=0x56439d8823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d8823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.970+0000 7f23df7fe640 1 -- 172.21.15.38:0/2941907693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439d875d40 con 0x56439d879b80 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23df7fe640 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d881e80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006ae0 tx=0x7f23d0013660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d334f3db263081f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23c57fa640 1 -- 172.21.15.38:0/2941907693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d002c070 con 0x56439d879b80 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23c57fa640 1 -- 172.21.15.38:0/2941907693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23d00125a0 con 0x56439d879b80 2024-04-25T13:04:39.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23c57fa640 1 -- 172.21.15.38:0/2941907693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0018070 con 0x56439d879b80 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 -- 172.21.15.38:0/2941907693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 msgr2=0x56439d881e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d881e80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006ae0 tx=0x7f23d0013660 comp rx=0 tx=0).stop 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 -- 172.21.15.38:0/2941907693 shutdown_connections 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87b710 0x56439d8823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d87ad50 0x56439d88b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/2941907693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d879b80 0x56439d881e80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 -- 172.21.15.38:0/2941907693 >> 172.21.15.38:0/2941907693 conn(0x56439d875070 msgr2=0x56439d8700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 -- 172.21.15.38:0/2941907693 shutdown_connections 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.971+0000 7f23e8a1a740 1 -- 172.21.15.38:0/2941907693 wait complete. 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 -- start start 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 Processor -- start 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d879b80 0x56439d886bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87ad50 0x56439d887110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 0x56439d88a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56439d892190 con 0x56439d87b710 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56439d891ac0 con 0x56439d87ad50 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56439d892860 con 0x56439d879b80 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23e8a1a740 10 client.0 _ll_register_callbacks cb 0x7f23d000c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 0x56439d88a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 0x56439d88a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44380/0 (socket says 172.21.15.38:44380) 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23dffff640 1 -- 172.21.15.38:0/989514149 learned_addr learned my addr 172.21.15.38:0/989514149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23deffd640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87ad50 0x56439d887110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.972+0000 7f23df7fe640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d879b80 0x56439d886bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23dffff640 1 -- 172.21.15.38:0/989514149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d879b80 msgr2=0x56439d886bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23dffff640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d879b80 0x56439d886bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23dffff640 1 -- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87ad50 msgr2=0x56439d887110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23dffff640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87ad50 0x56439d887110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23dffff640 1 -- 172.21.15.38:0/989514149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439d88a680 con 0x56439d87b710 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23df7fe640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d879b80 0x56439d886bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:04:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23dffff640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 0x56439d88a090 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009430 tx=0x7f23d802f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482a37f0f6d5cf49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23beffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23beffd640 1 -- 172.21.15.38:0/989514149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004850 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d00125a0 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23beffd640 1 -- 172.21.15.38:0/989514149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f23d8030b50 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23beffd640 1 -- 172.21.15.38:0/989514149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23d8030d50 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56439d8f6eb0 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23beffd640 1 -- 172.21.15.38:0/989514149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23d00125a0 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23beffd640 1 -- 172.21.15.38:0/989514149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23d802fd40 con 0x56439d87b710 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 20 client.15321 populate_metadata read hostname 'smithi038' 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d8fb680 0x56439d8f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 4 client.15321 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:04:39.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.973+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 --> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] -- command(tid 0: {"prefix": "status"}) -- 0x56439d7e64e0 con 0x56439d8fb680 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23df7fe640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d8fb680 0x56439d8f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23df7fe640 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d8fb680 0x56439d8f8190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e965dfae9c9019 server_cookie=41376222b13b4ef7 in_seq=0 out_seq=0 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23beffd640 10 client.15321 ms_handle_connect on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23beffd640 1 -- 172.21.15.38:0/989514149 <== mds.0 v2:172.21.15.104:6838/1961595450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x56439d7e64e0 con 0x56439d8fb680 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23beffd640 10 client.15321 handle_command_reply: tid=0 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23e8a1a740 1 client.15321 shutdown 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 msgr2=0x56439d88a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.974+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 0x56439d88a090 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009430 tx=0x7f23d802f530 comp rx=0 tx=0).stop 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 shutdown_connections 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56439d87b710 0x56439d88a090 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56439d87ad50 0x56439d887110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.104:6838/1961595450,v1:172.21.15.104:6839/1961595450] conn(0x56439d8fb680 0x56439d8f8190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 --2- 172.21.15.38:0/989514149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56439d879b80 0x56439d886bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23beffd640 0 client.15321 ms_handle_reset on v2:172.21.15.104:6838/1961595450 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 >> 172.21.15.38:0/989514149 conn(0x56439d875070 msgr2=0x56439d881d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 shutdown_connections 2024-04-25T13:04:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 1 -- 172.21.15.38:0/989514149 wait complete. 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e8a1a740 20 client.15321 trim_cache size 0 max 16384 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "d3627c2a-0302-11ef-bc93-c7b262605968", 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24518, 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 115.08114854, 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-04-25T13:04:39.981 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T13:04:39.982 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 119.331779875 2024-04-25T13:04:39.982 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:04:39.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f23c80a3bb0 msgr2=0x7f23c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f23c80a3bb0 0x7f23c80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23d804b000 tx=0x7f23d8048000 comp rx=0 tx=0).stop 2024-04-25T13:04:39.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 msgr2=0x7f23e01009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 0x7f23e01009e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c970 tx=0x7f23d400ce30 comp rx=0 tx=0).stop 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 shutdown_connections 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f23c80a3bb0 0x7f23c80a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e0107c90 0x7f23e0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e0105360 0x7f23e01009e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 --2- 172.21.15.38:0/2162759450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e0102a30 0x7f23e01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 >> 172.21.15.38:0/2162759450 conn(0x7f23e00fc440 msgr2=0x7f23e0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 shutdown_connections 2024-04-25T13:04:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:39.975+0000 7f23e5c2a640 1 -- 172.21.15.38:0/2162759450 wait complete. 2024-04-25T13:04:39.985 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:04:39.994 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:04:39.994 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:04:39.994 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:04:39.994 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T13:04:39.995 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:04:39.995 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:39.995 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 -- 172.21.15.38:0/2470319887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008101fc0 msgr2=0x7f50081023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 --2- 172.21.15.38:0/2470319887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008101fc0 0x7f50081023e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5004009900 tx=0x7f500402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 -- 172.21.15.38:0/2470319887 shutdown_connections 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 --2- 172.21.15.38:0/2470319887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5008103b70 0x7f500810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 --2- 172.21.15.38:0/2470319887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 0x7f5008103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 --2- 172.21.15.38:0/2470319887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008101fc0 0x7f50081023e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 -- 172.21.15.38:0/2470319887 >> 172.21.15.38:0/2470319887 conn(0x7f50080fd530 msgr2=0x7f50080ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 -- 172.21.15.38:0/2470319887 shutdown_connections 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.099+0000 7f501054f640 1 -- 172.21.15.38:0/2470319887 wait complete. 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 Processor -- start 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 -- start start 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5008101fc0 0x7f5008075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 0x7f5008074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 0x7f5008074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f500810dc80 con 0x7f5008103b70 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f500810db00 con 0x7f5008101fc0 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f501054f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f500810de00 con 0x7f50081031b0 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 0x7f5008074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 0x7f5008074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44394/0 (socket says 172.21.15.38:44394) 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 -- 172.21.15.38:0/3356427958 learned_addr learned my addr 172.21.15.38:0/3356427958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500e2eb640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5008101fc0 0x7f5008075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500daea640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 0x7f5008074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 -- 172.21.15.38:0/3356427958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 msgr2=0x7f5008074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 0x7f5008074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 -- 172.21.15.38:0/3356427958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5008101fc0 msgr2=0x7f5008075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5008101fc0 0x7f5008075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500eaec640 1 -- 172.21.15.38:0/3356427958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5008075110 con 0x7f5008103b70 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.100+0000 7f500daea640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 0x7f5008074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.101+0000 7f500eaec640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 0x7f5008074880 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f500000eaa0 tx=0x7f500000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.101+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500000cdb0 con 0x7f5008103b70 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.101+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5000005020 con 0x7f5008103b70 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.101+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000116d0 con 0x7f5008103b70 2024-04-25T13:04:40.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.101+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50081a23b0 con 0x7f5008103b70 2024-04-25T13:04:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.101+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50081a2980 con 0x7f5008103b70 2024-04-25T13:04:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.102+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5008109280 con 0x7f5008103b70 2024-04-25T13:04:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.104+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f50000051d0 con 0x7f5008103b70 2024-04-25T13:04:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.104+0000 7f4fff7fe640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4ff40a3cd0 0x7f4ff40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.104+0000 7f500e2eb640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4ff40a3cd0 0x7f4ff40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.104+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f50000d0a20 con 0x7f5008103b70 2024-04-25T13:04:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.105+0000 7f500e2eb640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4ff40a3cd0 0x7f4ff40a6190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5004009230 tx=0x7f5004031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.108+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5000016030 con 0x7f5008103b70 2024-04-25T13:04:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:40.348+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f500802d5c0 con 0x7f5008103b70 2024-04-25T13:04:40.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:40 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3784836780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:40.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:40 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3784836780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:40.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:40 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3784836780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:04:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.219+0000 7f4fff7fe640 1 -- 172.21.15.38:0/3356427958 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f500009d480 con 0x7f5008103b70 2024-04-25T13:04:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4ff40a3cd0 msgr2=0x7f4ff40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4ff40a3cd0 0x7f4ff40a6190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5004009230 tx=0x7f5004031510 comp rx=0 tx=0).stop 2024-04-25T13:04:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 msgr2=0x7f5008074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 0x7f5008074880 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f500000eaa0 tx=0x7f500000ef60 comp rx=0 tx=0).stop 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 shutdown_connections 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f4ff40a3cd0 0x7f4ff40a6190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008103b70 0x7f5008074880 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50081031b0 0x7f5008074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 --2- 172.21.15.38:0/3356427958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5008101fc0 0x7f5008075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.222+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 >> 172.21.15.38:0/3356427958 conn(0x7f50080fd530 msgr2=0x7f5008106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.223+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 shutdown_connections 2024-04-25T13:04:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.223+0000 7f501054f640 1 -- 172.21.15.38:0/3356427958 wait complete. 2024-04-25T13:04:41.242 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T13:04:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 -- 172.21.15.38:0/844216842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0100e60 msgr2=0x7f89b0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 --2- 172.21.15.38:0/844216842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0100e60 0x7f89b0101280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009900 tx=0x7f89a002a410 comp rx=0 tx=0).stop 2024-04-25T13:04:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 -- 172.21.15.38:0/844216842 shutdown_connections 2024-04-25T13:04:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 --2- 172.21.15.38:0/844216842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 0x7f89b0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 --2- 172.21.15.38:0/844216842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0102040 0x7f89b01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 --2- 172.21.15.38:0/844216842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0100e60 0x7f89b0101280 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 -- 172.21.15.38:0/844216842 >> 172.21.15.38:0/844216842 conn(0x7f89b00fc420 msgr2=0x7f89b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 -- 172.21.15.38:0/844216842 shutdown_connections 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.353+0000 7f89b62bb640 1 -- 172.21.15.38:0/844216842 wait complete. 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 Processor -- start 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 -- start start 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0100e60 0x7f89b006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 0x7f89b019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 0x7f89b019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89b010cb30 con 0x7f89b0102040 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89b010c9b0 con 0x7f89b0102a00 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b62bb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f89b010ccb0 con 0x7f89b0100e60 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 0x7f89b019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 0x7f89b019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44438/0 (socket says 172.21.15.38:44438) 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89af7fe640 1 -- 172.21.15.38:0/3511280713 learned_addr learned my addr 172.21.15.38:0/3511280713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89affff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0100e60 0x7f89b006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.354+0000 7f89b4858640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 0x7f89b019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b4858640 1 -- 172.21.15.38:0/3511280713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0100e60 msgr2=0x7f89b006a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b4858640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0100e60 0x7f89b006a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b4858640 1 -- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 msgr2=0x7f89b019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b4858640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 0x7f89b019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b4858640 1 -- 172.21.15.38:0/3511280713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b019fed0 con 0x7f89b0102a00 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89af7fe640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 0x7f89b019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89affff640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0100e60 0x7f89b006a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b4858640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 0x7f89b019f5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009fd0 tx=0x7f89a800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a800ce20 con 0x7f89b0102a00 2024-04-25T13:04:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f89a8005020 con 0x7f89b0102a00 2024-04-25T13:04:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b01a0160 con 0x7f89b0102a00 2024-04-25T13:04:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8010720 con 0x7f89b0102a00 2024-04-25T13:04:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.355+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b01a0660 con 0x7f89b0102a00 2024-04-25T13:04:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.356+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b0108200 con 0x7f89b0102a00 2024-04-25T13:04:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.357+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f89a8005560 con 0x7f89b0102a00 2024-04-25T13:04:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.358+0000 7f89ad7fa640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 0x7f89980a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.358+0000 7f89affff640 1 -- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 msgr2=0x7f89980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.358+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f89a80d0f80 con 0x7f89b0102a00 2024-04-25T13:04:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.358+0000 7f89affff640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 0x7f89980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.361+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89a809d960 con 0x7f89b0102a00 2024-04-25T13:04:41.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-mon[33233]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:41.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3356427958' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:04:41.489 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: ignoring --setuser ceph since I am not root 2024-04-25T13:04:41.490 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: ignoring --setgroup ceph since I am not root 2024-04-25T13:04:41.490 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-mgr[35107]: -- 172.21.15.178:0/1360041356 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4031365a0 con 0x55d40310b000 2024-04-25T13:04:41.490 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:41.402+0000 7f3809ecc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:04:41.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.553+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f89b002d5c0 con 0x7f89b0102a00 2024-04-25T13:04:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.559+0000 7f89affff640 1 -- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 msgr2=0x7f89980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.559+0000 7f89affff640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 0x7f89980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:41.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.563+0000 7f89ad7fa640 1 -- 172.21.15.38:0/3511280713 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f89a809d260 con 0x7f89b0102a00 2024-04-25T13:04:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 msgr2=0x7f89980a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 0x7f89980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 msgr2=0x7f89b019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 0x7f89b019f5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009fd0 tx=0x7f89a800ef10 comp rx=0 tx=0).stop 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 shutdown_connections 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f89980a3ca0 0x7f89980a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0102a00 0x7f89b019f5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b0102040 0x7f89b019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 --2- 172.21.15.38:0/3511280713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89b0100e60 0x7f89b006a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 >> 172.21.15.38:0/3511280713 conn(0x7f89b00fc420 msgr2=0x7f89b01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 shutdown_connections 2024-04-25T13:04:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.565+0000 7f89b62bb640 1 -- 172.21.15.38:0/3511280713 wait complete. 2024-04-25T13:04:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-mon[29400]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:41.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3356427958' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:04:41.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setuser ceph since I am not root 2024-04-25T13:04:41.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setgroup ceph since I am not root 2024-04-25T13:04:41.572 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:41.414+0000 7f9e62c30200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:04:41.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-mon[32726]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T13:04:41.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3356427958' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:04:41.574 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: ignoring --setuser ceph since I am not root 2024-04-25T13:04:41.574 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: ignoring --setgroup ceph since I am not root 2024-04-25T13:04:41.574 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-mgr[34118]: -- 172.21.15.104:0/3057376793 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575f51385a0 con 0x5575f510d000 2024-04-25T13:04:41.574 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:41.394+0000 7f0e4b331200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:04:41.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T13:04:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 -- 172.21.15.38:0/786028827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254102040 msgr2=0x7fe2541024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/786028827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254102040 0x7fe2541024c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe248009900 tx=0x7fe24802a410 comp rx=0 tx=0).stop 2024-04-25T13:04:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 -- 172.21.15.38:0/786028827 shutdown_connections 2024-04-25T13:04:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/786028827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe254102a00 0x7fe254109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/786028827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254102040 0x7fe2541024c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/786028827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe254100e60 0x7fe254101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 -- 172.21.15.38:0/786028827 >> 172.21.15.38:0/786028827 conn(0x7fe2540fc420 msgr2=0x7fe2540fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 -- 172.21.15.38:0/786028827 shutdown_connections 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.681+0000 7fe25a6e1640 1 -- 172.21.15.38:0/786028827 wait complete. 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 Processor -- start 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 -- start start 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 0x7fe25419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe254102040 0x7fe25419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe254102a00 0x7fe2541a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410cb50 con 0x7fe254100e60 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410c9d0 con 0x7fe254102a00 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe25a6e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410ccd0 con 0x7fe254102040 2024-04-25T13:04:41.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe253fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 0x7fe25419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe253fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 0x7fe25419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44462/0 (socket says 172.21.15.38:44462) 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe253fff640 1 -- 172.21.15.38:0/2795792690 learned_addr learned my addr 172.21.15.38:0/2795792690 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe258c7e640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe254102a00 0x7fe2541a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.682+0000 7fe2537fe640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe254102040 0x7fe25419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe253fff640 1 -- 172.21.15.38:0/2795792690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe254102040 msgr2=0x7fe25419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe253fff640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe254102040 0x7fe25419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe253fff640 1 -- 172.21.15.38:0/2795792690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe254102a00 msgr2=0x7fe2541a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe253fff640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe254102a00 0x7fe2541a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe253fff640 1 -- 172.21.15.38:0/2795792690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2541a2070 con 0x7fe254100e60 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe253fff640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 0x7fe25419a450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e9c0 tx=0x7fe24400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24400cd80 con 0x7fe254100e60 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe2440035f0 con 0x7fe254100e60 2024-04-25T13:04:41.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2541a2360 con 0x7fe254100e60 2024-04-25T13:04:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244010690 con 0x7fe254100e60 2024-04-25T13:04:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.683+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2541a2830 con 0x7fe254100e60 2024-04-25T13:04:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.684+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2541a2af0 con 0x7fe254100e60 2024-04-25T13:04:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.685+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe244003b30 con 0x7fe254100e60 2024-04-25T13:04:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.686+0000 7fe2517fa640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 0x7fe23c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.686+0000 7fe2537fe640 1 -- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 msgr2=0x7fe23c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.686+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe2440d07e0 con 0x7fe254100e60 2024-04-25T13:04:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.686+0000 7fe2537fe640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 0x7fe23c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.691+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe24409d1c0 con 0x7fe254100e60 2024-04-25T13:04:41.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.880+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe2541a32a0 con 0x7fe254100e60 2024-04-25T13:04:41.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.885+0000 7fe2517fa640 1 -- 172.21.15.38:0/2795792690 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fe24409cac0 con 0x7fe254100e60 2024-04-25T13:04:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe2537fe640 1 -- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 msgr2=0x7fe23c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe2537fe640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 0x7fe23c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 msgr2=0x7fe23c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 0x7fe23c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 msgr2=0x7fe25419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 0x7fe25419a450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e9c0 tx=0x7fe24400ee80 comp rx=0 tx=0).stop 2024-04-25T13:04:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 shutdown_connections 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fe23c0a3ca0 0x7fe23c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe254102a00 0x7fe2541a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe254102040 0x7fe25419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 --2- 172.21.15.38:0/2795792690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe254100e60 0x7fe25419a450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 >> 172.21.15.38:0/2795792690 conn(0x7fe2540fc420 msgr2=0x7fe2540fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.887+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 shutdown_connections 2024-04-25T13:04:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:41.888+0000 7fe25a6e1640 1 -- 172.21.15.38:0/2795792690 wait complete. 2024-04-25T13:04:41.897 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:41.610+0000 7f9e62c30200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:41.897 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:41.721+0000 7f9e62c30200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:04:41.898 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:04:41.921 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:04:41.922 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:04:41.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:41.922 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T13:04:41.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:41.938 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-04-25T13:04:41.980 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:41.576+0000 7f0e4b331200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:41.980 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:41.677+0000 7f0e4b331200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:04:41.980 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:41.981+0000 7f0e4b331200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:04:41.989 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:41.588+0000 7f3809ecc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:41.989 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:41.691+0000 7f3809ecc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:04:42.004 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-04-25T13:04:42.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:04:42.006 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'rbytes': False, 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T13:04:42.006 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:04:42.006 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:04:42.006 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:04:42.222 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:42.048+0000 7f9e62c30200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:04:42.223 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:42.139+0000 7f9e62c30200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:04:42.303 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:04:42.323 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:42.065+0000 7f0e4b331200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:04:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:42 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3356427958' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:04:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:42 smithi104 ceph-mon[32726]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T13:04:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:42 smithi104 ceph-mon[32726]: from='client.? ' entity='client.admin' 2024-04-25T13:04:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:42 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2795792690' entity='client.admin' 2024-04-25T13:04:42.489 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:42.001+0000 7f3809ecc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:04:42.489 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:42.088+0000 7f3809ecc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:04:42.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:42 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3356427958' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:04:42.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:42 smithi178 ceph-mon[33233]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T13:04:42.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:42 smithi178 ceph-mon[33233]: from='client.? ' entity='client.admin' 2024-04-25T13:04:42.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:42 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2795792690' entity='client.admin' 2024-04-25T13:04:42.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:42 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3356427958' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:04:42.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:42 smithi038 ceph-mon[29400]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T13:04:42.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:42 smithi038 ceph-mon[29400]: from='client.? ' entity='client.admin' 2024-04-25T13:04:42.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:42 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2795792690' entity='client.admin' 2024-04-25T13:04:43.557 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:43.226+0000 7f0e4b331200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:04:43.557 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:43.391+0000 7f0e4b331200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:43.557 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:43.557+0000 7f0e4b331200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:04:43.615 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:43.275+0000 7f3809ecc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:04:43.615 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:43.445+0000 7f3809ecc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:43.615 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:43.616+0000 7f3809ecc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:04:43.688 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:43.411+0000 7f9e62c30200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:04:43.688 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:43.597+0000 7f9e62c30200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:43.976 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:43.778+0000 7f9e62c30200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:04:44.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/2381243884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb4107980 msgr2=0x7fdcb4107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:44.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/2381243884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb4107980 0x7fdcb4107d60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009990 tx=0x7fdca4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/2381243884 shutdown_connections 2024-04-25T13:04:44.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/2381243884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb41022f0 0x7fdcb410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/2381243884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb4107980 0x7fdcb4107d60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/2381243884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 0x7fdcb4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/2381243884 >> 172.21.15.38:0/2381243884 conn(0x7fdcb40788b0 msgr2=0x7fdcb4078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:44.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.083+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/2381243884 shutdown_connections 2024-04-25T13:04:44.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.084+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/2381243884 wait complete. 2024-04-25T13:04:44.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.084+0000 7fdcb9cb9700 1 Processor -- start 2024-04-25T13:04:44.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.084+0000 7fdcb9cb9700 1 -- start start 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb9cb9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 0x7fdcb419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb9cb9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 0x7fdcb419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb9cb9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb4107980 0x7fdcb41a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb9cb9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4111c50 con 0x7fdcb41022f0 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb9cb9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4111ad0 con 0x7fdcb4107980 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb9cb9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4111dd0 con 0x7fdcb4101930 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 0x7fdcb419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 0x7fdcb419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44468/0 (socket says 172.21.15.38:44468) 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 0x7fdcb419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb2ffd700 1 -- 172.21.15.38:0/1467937546 learned_addr learned my addr 172.21.15.38:0/1467937546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb3fff700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb4107980 0x7fdcb41a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 -- 172.21.15.38:0/1467937546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb4107980 msgr2=0x7fdcb41a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb4107980 0x7fdcb41a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 -- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 msgr2=0x7fdcb419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 0x7fdcb419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 -- 172.21.15.38:0/1467937546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb41a55d0 con 0x7fdcb4101930 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb2ffd700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 0x7fdcb419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb37fe700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 0x7fdcb419a010 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdca40098e0 tx=0x7fdca40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.085+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4027070 con 0x7fdcb4101930 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.086+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdca400b610 con 0x7fdcb4101930 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.086+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb41a5800 con 0x7fdcb4101930 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.086+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40173f0 con 0x7fdcb4101930 2024-04-25T13:04:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.086+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb41a5c50 con 0x7fdcb4101930 2024-04-25T13:04:44.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.087+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb4109ea0 con 0x7fdcb4101930 2024-04-25T13:04:44.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.088+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fdca402a070 con 0x7fdcb4101930 2024-04-25T13:04:44.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.088+0000 7fdcb0ff9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 0x7fdca00a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:44.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.088+0000 7fdcb2ffd700 1 -- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 msgr2=0x7fdca00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:44.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.088+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdca40d1360 con 0x7fdcb4101930 2024-04-25T13:04:44.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.088+0000 7fdcb2ffd700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 0x7fdca00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:44.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.093+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdca405eff0 con 0x7fdcb4101930 2024-04-25T13:04:44.134 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:43.840+0000 7f0e4b331200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:04:44.134 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:43.933+0000 7f0e4b331200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:04:44.134 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:44.135+0000 7f0e4b331200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:04:44.203 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:43.903+0000 7f3809ecc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:04:44.203 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:43.998+0000 7f3809ecc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:04:44.203 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:44.203+0000 7f3809ecc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:04:44.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.289+0000 7fdcb2ffd700 1 -- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 msgr2=0x7fdca00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:44.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.289+0000 7fdcb2ffd700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 0x7fdca00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:44.289 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:44.091+0000 7f9e62c30200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:04:44.289 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:44.189+0000 7f9e62c30200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:04:44.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.354+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdcb402df80 con 0x7fdcb4101930 2024-04-25T13:04:44.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.355+0000 7fdcb0ff9700 1 -- 172.21.15.38:0/1467937546 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fdca4013070 con 0x7fdcb4101930 2024-04-25T13:04:44.354 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:04:44.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 msgr2=0x7fdca00a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:44.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 0x7fdca00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 msgr2=0x7fdcb419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:44.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 0x7fdcb419a010 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdca40098e0 tx=0x7fdca40057c0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 shutdown_connections 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7fdca00a3c90 0x7fdca00a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb4107980 0x7fdcb41a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcb4101930 0x7fdcb419a010 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 --2- 172.21.15.38:0/1467937546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb41022f0 0x7fdcb419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 >> 172.21.15.38:0/1467937546 conn(0x7fdcb40788b0 msgr2=0x7fdcb4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 shutdown_connections 2024-04-25T13:04:44.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:44.357+0000 7fdcb9cb9700 1 -- 172.21.15.38:0/1467937546 wait complete. 2024-04-25T13:04:44.406 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:44.325+0000 7f0e4b331200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:44.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:44 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1467937546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:04:44.489 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:44.397+0000 7f3809ecc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:44.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:44 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1467937546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:04:44.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:44.406+0000 7f9e62c30200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:04:44.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:44 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1467937546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:04:45.071 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:44.609+0000 7f9e62c30200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:04:45.157 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:04:45.157 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:04:45.157 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T13:04:45.157 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:04:45.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:04:45.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:04:45.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:04:45.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:04:45.158 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:04:45.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:45.158 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T13:04:45.174 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:04:45.174 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:04:45.175 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42110sec preferred_lft 42110sec 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:04:45.176 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T13:04:45.176 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:04:45.176 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:04:45.176 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T13:04:45.177 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:04:45.177 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-04-25T13:04:45.177 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-04-25T13:04:45.177 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:04:45.177 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T13:04:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:04:45.323 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:44.916+0000 7f0e4b331200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:04:45.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:45.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:45.371 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:04:45.379 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:45.245+0000 7f9e62c30200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:04:45.396 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T13:04:45.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:45.398 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T13:04:45.423 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:44.999+0000 7f3809ecc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:04:45.471 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T13:04:45.471 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:04:45.471 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T13:04:45.471 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T13:04:45.471 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:04:45.472 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:04:45.472 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T13:04:45.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T13:04:45.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T13:04:45.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T13:04:45.472 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T13:04:45.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:04:45.622 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:45.332+0000 7f0e4b331200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:04:45.622 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:45.426+0000 7f0e4b331200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:04:45.622 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:45.623+0000 7f0e4b331200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:04:45.723 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:45.425+0000 7f3809ecc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:04:45.723 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:45.522+0000 7f3809ecc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:04:45.723 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:45.724+0000 7f3809ecc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:04:45.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:45.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:45.736 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T13:04:45.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:45.752 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-04-25T13:04:45.808 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:04:45.808 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T13:04:45.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:04:45.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:04:45.809 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T13:04:45.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:04:45.907 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:45.714+0000 7f9e62c30200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:04:45.907 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:45.819+0000 7f9e62c30200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:04:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:45.929 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:04:45.929 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:04:45.929 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T13:04:45.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:04:45.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:04:45.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:04:45.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:04:45.930 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:04:45.930 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T13:04:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:04:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:46.057 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:04:46.057 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T13:04:46.057 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T13:04:46.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:04:46.058 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T13:04:46.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:04:46.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:46.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:04:46.136 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:04:46.151 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:04:46.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:04:46.151 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:04:46.207 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:04:46.207 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:04:46.207 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/f2b81406-7843-4642-928a-a1fc987285dd 2024-04-25T13:04:46.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:46.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/f2b81406-7843-4642-928a-a1fc987285dd /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T13:04:46.270 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:46.044+0000 7f9e62c30200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:04:46.308 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T13:04:46.308 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-25T13:04:46.435 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T13:04:46.435 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-25T13:04:46.435 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.104:3300,172.21.15.178:3300" 2024-04-25T13:04:46.436 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with new device syntax: 0@d3627c2a-0302-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/f2b81406-7843-4642-928a-a1fc987285dd 2024-04-25T13:04:46.436 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.38:3300/172.21.15.104:3300/172.21.15.178:3300" will pass to kernel 2024-04-25T13:04:46.437 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:04:46.437 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:04:46.437 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:04:46.437 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:04:46.437 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:04:46.440 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:04:46.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:04:46.440 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:04:46.462 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:46.136+0000 7f0e4b331200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:04:46.463 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:46.225+0000 7f0e4b331200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:04:46.508 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:04:46.582 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:46.249+0000 7f3809ecc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:04:46.582 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:46.340+0000 7f3809ecc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:04:46.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:04:46.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:46.607 INFO:teuthology.orchestra.run.smithi038.stdout:client24973 2024-04-25T13:04:46.607 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T13:04:46.608 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:04:46.625 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:04:46.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:04:46.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:46.725 INFO:teuthology.orchestra.run.smithi038.stdout:client24973 2024-04-25T13:04:46.725 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:04:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:04:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:46.783 INFO:teuthology.orchestra.run.smithi038.stdout:client24973 2024-04-25T13:04:46.784 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d3627c2a-0302-11ef-bc93-c7b262605968.client24973/status') 2024-04-25T13:04:46.798 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:46.463+0000 7f0e4b331200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:04:46.798 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:46.593+0000 7f0e4b331200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:04:46.798 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:46.685+0000 7f0e4b331200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:04:46.798 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:46.799+0000 7f0e4b331200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:46.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:04:46.848 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.24973 (3)172.21.15.38:0/3038550408 2024-04-25T13:04:46.848 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-04-25T13:04:46.848 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-04-25T13:04:46.848 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-04-25T13:04:46.848 INFO:teuthology.orchestra.run.smithi038.stderr:71 bytes copied, 2.9098e-05 s, 2.4 MB/s 2024-04-25T13:04:46.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:04:46.856 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:04:46.866 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:04:46.866 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:46.866 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T13:04:46.888 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:46.614+0000 7f9e62c30200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:04:46.888 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:46.717+0000 7f9e62c30200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:04:46.927 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:46.583+0000 7f3809ecc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:04:46.927 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:46.716+0000 7f3809ecc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:04:46.927 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:46.812+0000 7f3809ecc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:04:46.927 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:46.928+0000 7f3809ecc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:46.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 -- 172.21.15.38:0/1503537756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 msgr2=0x7f48cc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:46.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 --2- 172.21.15.38:0/1503537756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc1024c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009900 tx=0x7f48c002a410 comp rx=0 tx=0).stop 2024-04-25T13:04:46.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 -- 172.21.15.38:0/1503537756 shutdown_connections 2024-04-25T13:04:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 --2- 172.21.15.38:0/1503537756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc102a00 0x7f48cc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 --2- 172.21.15.38:0/1503537756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc1024c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 --2- 172.21.15.38:0/1503537756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc100e60 0x7f48cc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 -- 172.21.15.38:0/1503537756 >> 172.21.15.38:0/1503537756 conn(0x7f48cc0fc420 msgr2=0x7f48cc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 -- 172.21.15.38:0/1503537756 shutdown_connections 2024-04-25T13:04:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 -- 172.21.15.38:0/1503537756 wait complete. 2024-04-25T13:04:46.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.987+0000 7f48d11e5640 1 Processor -- start 2024-04-25T13:04:46.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 -- start start 2024-04-25T13:04:46.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc100e60 0x7f48cc10b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:46.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:46.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc102a00 0x7f48cc10a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:46.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10dc40 con 0x7f48cc102040 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10dac0 con 0x7f48cc102a00 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48d11e5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10ddc0 con 0x7f48cc100e60 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc109ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44500/0 (socket says 172.21.15.38:44500) 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48cad9d640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc100e60 0x7f48cc10b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 -- 172.21.15.38:0/2192892877 learned_addr learned my addr 172.21.15.38:0/2192892877 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48cb59e640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc102a00 0x7f48cc10a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 -- 172.21.15.38:0/2192892877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc100e60 msgr2=0x7f48cc10b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc100e60 0x7f48cc10b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 -- 172.21.15.38:0/2192892877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc102a00 msgr2=0x7f48cc10a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc102a00 0x7f48cc10a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48ca59c640 1 -- 172.21.15.38:0/2192892877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc10ac70 con 0x7f48cc102040 2024-04-25T13:04:46.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48cb59e640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc102a00 0x7f48cc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.988+0000 7f48cad9d640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc100e60 0x7f48cc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.989+0000 7f48ca59c640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc109ea0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005940 tx=0x7f48c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.989+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c003f030 con 0x7f48cc102040 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.989+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc1a22f0 con 0x7f48cc102040 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.989+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f48c0003cd0 con 0x7f48cc102040 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.989+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0035070 con 0x7f48cc102040 2024-04-25T13:04:46.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.989+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1a2890 con 0x7f48cc102040 2024-04-25T13:04:46.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.990+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc108200 con 0x7f48cc102040 2024-04-25T13:04:46.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.991+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f48c0042030 con 0x7f48cc102040 2024-04-25T13:04:46.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.992+0000 7f48b3fff640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 0x7f48b40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:46.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.992+0000 7f48cad9d640 1 -- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 msgr2=0x7f48b40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:46.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.992+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f48c00f08a0 con 0x7f48cc102040 2024-04-25T13:04:46.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:46.992+0000 7f48cad9d640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 0x7f48b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.001+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48c00bcc30 con 0x7f48cc102040 2024-04-25T13:04:47.141 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:46.994+0000 7f9e62c30200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:04:47.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.193+0000 7f48cad9d640 1 -- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 msgr2=0x7f48b40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:47.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.193+0000 7f48cad9d640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 0x7f48b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:47.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.231+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f48cc062b50 con 0x7f48cc102040 2024-04-25T13:04:47.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.283+0000 7f48b3fff640 1 -- 172.21.15.38:0/2192892877 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f48c00c0f30 con 0x7f48cc102040 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 msgr2=0x7f48b40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 0x7f48b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 msgr2=0x7f48cc109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc109ea0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005940 tx=0x7f48c00042f0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 shutdown_connections 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f48b40a3ca0 0x7f48b40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48cc102a00 0x7f48cc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc102040 0x7f48cc109ea0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 --2- 172.21.15.38:0/2192892877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48cc100e60 0x7f48cc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 >> 172.21.15.38:0/2192892877 conn(0x7f48cc0fc420 msgr2=0x7f48cc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:47.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 shutdown_connections 2024-04-25T13:04:47.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.286+0000 7f48d11e5640 1 -- 172.21.15.38:0/2192892877 wait complete. 2024-04-25T13:04:47.297 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:04:47.308 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:04:47.334 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:04:47.334 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:47.335 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.444+0000 7f9cb59db640 1 -- 172.21.15.38:0/1545242963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 msgr2=0x7f9cb010a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 --2- 172.21.15.38:0/1545242963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 0x7f9cb010a120 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009ca0 tx=0x7f9ca802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 -- 172.21.15.38:0/1545242963 shutdown_connections 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 --2- 172.21.15.38:0/1545242963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 0x7f9cb010a120 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 --2- 172.21.15.38:0/1545242963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cb0105400 0x7f9cb01077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 --2- 172.21.15.38:0/1545242963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cb0102ad0 0x7f9cb0104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 -- 172.21.15.38:0/1545242963 >> 172.21.15.38:0/1545242963 conn(0x7f9cb00fc440 msgr2=0x7f9cb00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 -- 172.21.15.38:0/1545242963 shutdown_connections 2024-04-25T13:04:47.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 -- 172.21.15.38:0/1545242963 wait complete. 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 Processor -- start 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.445+0000 7f9cb59db640 1 -- start start 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cb59db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cb0102ad0 0x7f9cb0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cb59db640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cb0105400 0x7f9cb0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cb59db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 0x7f9cb019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cb59db640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb010dc50 con 0x7f9cb0107d30 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cb59db640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb010dad0 con 0x7f9cb0105400 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cb59db640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb010ddd0 con 0x7f9cb0102ad0 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 0x7f9cb019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cb0102ad0 0x7f9cb0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 0x7f9cb019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44510/0 (socket says 172.21.15.38:44510) 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caf7fe640 1 -- 172.21.15.38:0/2992713620 learned_addr learned my addr 172.21.15.38:0/2992713620 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:47.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cb0105400 0x7f9cb0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cb0105400 msgr2=0x7f9cb0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cb0105400 0x7f9cb0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 msgr2=0x7f9cb019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cb0107d30 0x7f9cb019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 -- 172.21.15.38:0/2992713620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb019fe20 con 0x7f9cb0102ad0 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cb0105400 0x7f9cb0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:04:47.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.446+0000 7f9caeffd640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cb0102ad0 0x7f9cb0198200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000e9f0 tx=0x7f9ca000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.447+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000ccc0 con 0x7f9cb0102ad0 2024-04-25T13:04:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.447+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb01a0110 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.447+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ca0010900 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.447+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0018920 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.447+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb01a06b0 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.448+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb00fdf30 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.449+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f9ca0010430 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.449+0000 7f9cb49d9640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 0x7f9c980a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.450+0000 7f9cae7fc640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 msgr2=0x7f9c980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.450+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9ca00d0750 con 0x7f9cb0102ad0 2024-04-25T13:04:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.450+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 0x7f9c980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:04:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.454+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ca0010720 con 0x7f9cb0102ad0 2024-04-25T13:04:47.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:47 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2192892877' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:04:47.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:47.142+0000 7f9e62c30200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:04:47.571 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:47.245+0000 7f9e62c30200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:04:47.572 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:47.373+0000 7f9e62c30200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:04:47.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:47 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2192892877' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:04:47.574 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:47.096+0000 7f0e4b331200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:04:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.651+0000 7f9cae7fc640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 msgr2=0x7f9c980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.651+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 0x7f9c980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:04:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:47.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 --> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9cb0062390 con 0x7f9c980a3ca0 2024-04-25T13:04:47.739 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:47.241+0000 7f3809ecc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:04:47.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:47 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2192892877' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:04:48.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:48.052+0000 7f9cae7fc640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 msgr2=0x7f9c980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1333793113 2024-04-25T13:04:48.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:48.052+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 0x7f9c980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:04:48.071 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:47.693+0000 7f9e62c30200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:04:48.073 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:47.690+0000 7f0e4b331200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:04:48.073 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:04:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y[34113]: 2024-04-25T13:04:47.904+0000 7f0e4b331200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:04:48.239 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:47.848+0000 7f3809ecc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:04:48.239 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z[35103]: 2024-04-25T13:04:48.066+0000 7f3809ecc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:04:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:48.310+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f9ca0099150 con 0x7f9cb0102ad0 2024-04-25T13:04:48.563 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.335+0000 7f9e62c30200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:04:48.563 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2192892877' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: Standby manager daemon y restarted 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: Standby manager daemon y started 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: Standby manager daemon z restarted 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: Standby manager daemon z started 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:04:48.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-mon[29400]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:04:48.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2192892877' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: Standby manager daemon y restarted 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: Standby manager daemon y started 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: Standby manager daemon z restarted 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: Standby manager daemon z started 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:04:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:48 smithi104 ceph-mon[32726]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:04:48.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:48.580+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f9ca00ce2d0 con 0x7f9cb0102ad0 2024-04-25T13:04:48.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:48.581+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 msgr2=0x7f9c980a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:04:48.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:48.581+0000 7f9cb49d9640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1333793113,v1:172.21.15.38:6801/1333793113] conn(0x7f9c980a3ca0 0x7f9c980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:48.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2192892877' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: Standby manager daemon y restarted 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: Standby manager daemon y started 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.104:0/3081046040' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: Standby manager daemon z restarted 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: Standby manager daemon z started 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:04:48.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:04:48.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:48 smithi178 ceph-mon[33233]: from='mgr.? 172.21.15.178:0/999262707' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:04:48.821 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.563+0000 7f9e62c30200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:04:49.305 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc1ddc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.305 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc1ddc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.305 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc1ddc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.305 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc1ddc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.305 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc1ddc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.306 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc45e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.306 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc45e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.306 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc45e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.306 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc45e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.306 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:04:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:04:48.821+0000 7f9dc45e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:04:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: Active manager daemon x restarted 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: Activating manager daemon x 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: mgrmap e21: x(active, starting, since 0.0148277s), standbys: z, y 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:04:49.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:04:49.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:04:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: Manager daemon x is now available 2024-04-25T13:04:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:04:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:49 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:04:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T13:04:49.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: Active manager daemon x restarted 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: Activating manager daemon x 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: mgrmap e21: x(active, starting, since 0.0148277s), standbys: z, y 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:04:49.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:04:49.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: Manager daemon x is now available 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:04:49.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:49 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:04:49.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.587+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mon.1 v2:172.21.15.178:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f9ca0078b80 con 0x7f9cb0102ad0 2024-04-25T13:04:49.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.588+0000 7f9cb49d9640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f9c98035890 0x7f9c98035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:49.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.588+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 --> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9ca0011e50 con 0x7f9c98035890 2024-04-25T13:04:49.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.592+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f9c98035890 0x7f9c98035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:49.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.595+0000 7f9cae7fc640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f9c98035890 0x7f9c98035c70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a010 tx=0x7f9ca4005f90 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.664+0000 7f9cb49d9640 1 -- 172.21.15.38:0/2992713620 <== mgr.15345 v2:172.21.15.38:6800/1710923873 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9ca0011e50 con 0x7f9c98035890 2024-04-25T13:04:49.664 INFO:teuthology.orchestra.run.smithi038.stdout:Schedule set for path / 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f9c98035890 msgr2=0x7f9c98035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f9c98035890 0x7f9c98035c70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a010 tx=0x7f9ca4005f90 comp rx=0 tx=0).stop 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cb0102ad0 msgr2=0x7f9cb0198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 --2- 172.21.15.38:0/2992713620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cb0102ad0 0x7f9cb0198200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000e9f0 tx=0x7f9ca000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9caf7fe640 1 -- 172.21.15.38:0/2992713620 reap_dead start 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 shutdown_connections 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 >> 172.21.15.38:0/2992713620 conn(0x7f9cb00fc440 msgr2=0x7f9cb0103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 shutdown_connections 2024-04-25T13:04:49.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.666+0000 7f9cb59db640 1 -- 172.21.15.38:0/2992713620 wait complete. 2024-04-25T13:04:49.680 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T13:04:49.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: Active manager daemon x restarted 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: Activating manager daemon x 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: mgrmap e21: x(active, starting, since 0.0148277s), standbys: z, y 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:04:49.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:04:49.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: Manager daemon x is now available 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:49.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:04:49.743 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:49 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:04:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 -- 172.21.15.38:0/3850643699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190100e60 msgr2=0x7f0190101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 --2- 172.21.15.38:0/3850643699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190100e60 0x7f0190101280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 -- 172.21.15.38:0/3850643699 shutdown_connections 2024-04-25T13:04:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 --2- 172.21.15.38:0/3850643699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0190102a00 0x7f0190109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 --2- 172.21.15.38:0/3850643699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0190102040 0x7f01901024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 --2- 172.21.15.38:0/3850643699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190100e60 0x7f0190101280 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 -- 172.21.15.38:0/3850643699 >> 172.21.15.38:0/3850643699 conn(0x7f01900fc420 msgr2=0x7f01900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 -- 172.21.15.38:0/3850643699 shutdown_connections 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 -- 172.21.15.38:0/3850643699 wait complete. 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 Processor -- start 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.795+0000 7f019575d640 1 -- start start 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0190100e60 0x7f0190198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0190102040 0x7f0190198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 0x7f019019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f019010dc40 con 0x7f0190102a00 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f019010dac0 con 0x7f0190102040 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f019010ddc0 con 0x7f0190100e60 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 0x7f019019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 0x7f019019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41724/0 (socket says 172.21.15.38:41724) 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 -- 172.21.15.38:0/97572680 learned_addr learned my addr 172.21.15.38:0/97572680 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:49.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018e7fc640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0190102040 0x7f0190198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018effd640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0190100e60 0x7f0190198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 -- 172.21.15.38:0/97572680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0190100e60 msgr2=0x7f0190198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0190100e60 0x7f0190198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 -- 172.21.15.38:0/97572680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0190102040 msgr2=0x7f0190198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0190102040 0x7f0190198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 -- 172.21.15.38:0/97572680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019019fe70 con 0x7f0190102a00 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f018f7fe640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 0x7f019019f800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f018800efd0 tx=0x7f018800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188003b70 con 0x7f0190102a00 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0188003d10 con 0x7f0190102a00 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01901a0160 con 0x7f0190102a00 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188010990 con 0x7f0190102a00 2024-04-25T13:04:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.796+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01901a06b0 con 0x7f0190102a00 2024-04-25T13:04:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.797+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0190108200 con 0x7f0190102a00 2024-04-25T13:04:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.798+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0188004020 con 0x7f0190102a00 2024-04-25T13:04:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.799+0000 7f0177fff640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f01780a3ae0 0x7f01780a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.799+0000 7f018effd640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f01780a3ae0 0x7f01780a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.799+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f01880d04c0 con 0x7f0190102a00 2024-04-25T13:04:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.800+0000 7f018effd640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f01780a3ae0 0x7f01780a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0184009230 tx=0x7f0184031510 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:49.804+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f018809d080 con 0x7f0190102a00 2024-04-25T13:04:50.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.019+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 --> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f0190062430 con 0x7f01780a3ae0 2024-04-25T13:04:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.078+0000 7f0177fff640 1 -- 172.21.15.38:0/97572680 <== mgr.15345 v2:172.21.15.38:6800/1710923873 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f0190062430 con 0x7f01780a3ae0 2024-04-25T13:04:50.078 INFO:teuthology.orchestra.run.smithi038.stdout:Retention added to path / 2024-04-25T13:04:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f01780a3ae0 msgr2=0x7f01780a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f01780a3ae0 0x7f01780a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0184009230 tx=0x7f0184031510 comp rx=0 tx=0).stop 2024-04-25T13:04:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 msgr2=0x7f019019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 0x7f019019f800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f018800efd0 tx=0x7f018800c6a0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 shutdown_connections 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0190102a00 0x7f019019f800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0190102040 0x7f0190198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f01780a3ae0 0x7f01780a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 --2- 172.21.15.38:0/97572680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0190100e60 0x7f0190198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.080+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 >> 172.21.15.38:0/97572680 conn(0x7f01900fc420 msgr2=0x7f01901053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.081+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 shutdown_connections 2024-04-25T13:04:50.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.081+0000 7f019575d640 1 -- 172.21.15.38:0/97572680 wait complete. 2024-04-25T13:04:50.091 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:04:50.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 -- 172.21.15.38:0/139891533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1320102ba0 msgr2=0x7f1320104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 --2- 172.21.15.38:0/139891533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1320102ba0 0x7f1320104f90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1314009900 tx=0x7f131402a410 comp rx=0 tx=0).stop 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 -- 172.21.15.38:0/139891533 shutdown_connections 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 --2- 172.21.15.38:0/139891533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1320107e00 0x7f132010a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 --2- 172.21.15.38:0/139891533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13201054d0 0x7f13201078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 --2- 172.21.15.38:0/139891533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1320102ba0 0x7f1320104f90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 -- 172.21.15.38:0/139891533 >> 172.21.15.38:0/139891533 conn(0x7f13200fc4d0 msgr2=0x7f13200fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 -- 172.21.15.38:0/139891533 shutdown_connections 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.195+0000 7f13270d9640 1 -- 172.21.15.38:0/139891533 wait complete. 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 Processor -- start 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 -- start start 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1320102ba0 0x7f1320198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 0x7f1320198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1320107e00 0x7f132019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f132010dc60 con 0x7f13201054d0 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f132010dae0 con 0x7f1320102ba0 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f13270d9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f132010dde0 con 0x7f1320107e00 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 0x7f1320198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 0x7f1320198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41734/0 (socket says 172.21.15.38:41734) 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 -- 172.21.15.38:0/759374878 learned_addr learned my addr 172.21.15.38:0/759374878 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f1324e75640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1320102ba0 0x7f1320198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f1325676640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1320107e00 0x7f132019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 -- 172.21.15.38:0/759374878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1320107e00 msgr2=0x7f132019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1320107e00 0x7f132019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 -- 172.21.15.38:0/759374878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1320102ba0 msgr2=0x7f1320198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1320102ba0 0x7f1320198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 -- 172.21.15.38:0/759374878 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132019fe50 con 0x7f13201054d0 2024-04-25T13:04:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.196+0000 7f131ffff640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 0x7f1320198770 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f130c005fd0 tx=0x7f130c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:50.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.197+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c017070 con 0x7f13201054d0 2024-04-25T13:04:50.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.197+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f130c009c30 con 0x7f13201054d0 2024-04-25T13:04:50.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.197+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13201a0140 con 0x7f13201054d0 2024-04-25T13:04:50.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.197+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c003de0 con 0x7f13201054d0 2024-04-25T13:04:50.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.197+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13201a0690 con 0x7f13201054d0 2024-04-25T13:04:50.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.198+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13200fdfc0 con 0x7f13201054d0 2024-04-25T13:04:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.199+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f130c003f80 con 0x7f13201054d0 2024-04-25T13:04:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.199+0000 7f131dffb640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f13100a3a60 0x7f13100a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.200+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f130c0cfd90 con 0x7f13201054d0 2024-04-25T13:04:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.200+0000 7f1324e75640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f13100a3a60 0x7f13100a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.200+0000 7f1324e75640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f13100a3a60 0x7f13100a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f1314031510 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:50.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.204+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f130c020030 con 0x7f13201054d0 2024-04-25T13:04:50.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.420+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 --> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1320062390 con 0x7f13100a3a60 2024-04-25T13:04:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.428+0000 7f131dffb640 1 -- 172.21.15.38:0/759374878 <== mgr.15345 v2:172.21.15.38:6800/1710923873 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1320062390 con 0x7f13100a3a60 2024-04-25T13:04:50.428 INFO:teuthology.orchestra.run.smithi038.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:04:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f13100a3a60 msgr2=0x7f13100a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f13100a3a60 0x7f13100a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f1314031510 comp rx=0 tx=0).stop 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 msgr2=0x7f1320198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 0x7f1320198770 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f130c005fd0 tx=0x7f130c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 shutdown_connections 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f13100a3a60 0x7f13100a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1320107e00 0x7f132019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13201054d0 0x7f1320198770 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 --2- 172.21.15.38:0/759374878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1320102ba0 0x7f1320198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 >> 172.21.15.38:0/759374878 conn(0x7f13200fc4d0 msgr2=0x7f13201034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 shutdown_connections 2024-04-25T13:04:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.430+0000 7f13270d9640 1 -- 172.21.15.38:0/759374878 wait complete. 2024-04-25T13:04:50.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:04:50.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 -- 172.21.15.38:0/483752669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102a20 msgr2=0x7fe528109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 --2- 172.21.15.38:0/483752669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102a20 0x7fe528109340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe520009ca0 tx=0x7fe52002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 -- 172.21.15.38:0/483752669 shutdown_connections 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 --2- 172.21.15.38:0/483752669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102a20 0x7fe528109340 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 --2- 172.21.15.38:0/483752669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe528102060 0x7fe5281024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 --2- 172.21.15.38:0/483752669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe528100e80 0x7fe5281012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 -- 172.21.15.38:0/483752669 >> 172.21.15.38:0/483752669 conn(0x7fe5280fc460 msgr2=0x7fe5280fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 -- 172.21.15.38:0/483752669 shutdown_connections 2024-04-25T13:04:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 -- 172.21.15.38:0/483752669 wait complete. 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.563+0000 7fe52dd44640 1 Processor -- start 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 -- start start 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe528100e80 0x7fe5281970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 0x7fe528197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe528102a20 0x7fe52819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810cb60 con 0x7fe528102060 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810c9e0 con 0x7fe528102a20 2024-04-25T13:04:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe52dd44640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810cce0 con 0x7fe528100e80 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 0x7fe528197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe5277fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe528100e80 0x7fe5281970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 0x7fe528197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41752/0 (socket says 172.21.15.38:41752) 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 -- 172.21.15.38:0/3364778196 learned_addr learned my addr 172.21.15.38:0/3364778196 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe527fff640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe528102a20 0x7fe52819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 -- 172.21.15.38:0/3364778196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe528100e80 msgr2=0x7fe5281970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe528100e80 0x7fe5281970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 -- 172.21.15.38:0/3364778196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe528102a20 msgr2=0x7fe52819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe528102a20 0x7fe52819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.564+0000 7fe526ffd640 1 -- 172.21.15.38:0/3364778196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52819ed00 con 0x7fe528102060 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.565+0000 7fe526ffd640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 0x7fe528197620 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e9f0 tx=0x7fe51800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.565+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800cd80 con 0x7fe528102060 2024-04-25T13:04:50.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.565+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe518010900 con 0x7fe528102060 2024-04-25T13:04:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.565+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52819eff0 con 0x7fe528102060 2024-04-25T13:04:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.565+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180188d0 con 0x7fe528102060 2024-04-25T13:04:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.565+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52819f4b0 con 0x7fe528102060 2024-04-25T13:04:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.566+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5281081c0 con 0x7fe528102060 2024-04-25T13:04:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.567+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fe518010ab0 con 0x7fe528102060 2024-04-25T13:04:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.568+0000 7fe524ff9640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fe5100a3a60 0x7fe5100a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:04:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.568+0000 7fe5277fe640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fe5100a3a60 0x7fe5100a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:04:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.568+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe5180d0490 con 0x7fe528102060 2024-04-25T13:04:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.569+0000 7fe5277fe640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fe5100a3a60 0x7fe5100a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe528101bd0 tx=0x7fe51c005f90 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:04:50.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.573+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe51809ca00 con 0x7fe528102060 2024-04-25T13:04:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.597+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fe518099070 con 0x7fe528102060 2024-04-25T13:04:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:50 smithi038 ceph-mon[29400]: mgrmap e22: x(active, since 1.0236s), standbys: z, y 2024-04-25T13:04:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:50 smithi038 ceph-mon[29400]: [25/Apr/2024:13:04:49] ENGINE Bus STARTING 2024-04-25T13:04:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:50 smithi038 ceph-mon[29400]: [25/Apr/2024:13:04:49] ENGINE Serving on https://172.21.15.38:7150 2024-04-25T13:04:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:50 smithi038 ceph-mon[29400]: [25/Apr/2024:13:04:49] ENGINE Serving on http://172.21.15.38:8765 2024-04-25T13:04:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:50 smithi038 ceph-mon[29400]: [25/Apr/2024:13:04:49] ENGINE Bus STARTED 2024-04-25T13:04:50.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:50 smithi038 ceph-mon[29400]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:50.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.831+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 --> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe528062390 con 0x7fe5100a3a60 2024-04-25T13:04:50.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.838+0000 7fe524ff9640 1 -- 172.21.15.38:0/3364778196 <== mgr.15345 v2:172.21.15.38:6800/1710923873 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe528062390 con 0x7fe5100a3a60 2024-04-25T13:04:50.837 INFO:teuthology.orchestra.run.smithi038.stdout:/ 1m 6m3h 2024-04-25T13:04:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fe5100a3a60 msgr2=0x7fe5100a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fe5100a3a60 0x7fe5100a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe528101bd0 tx=0x7fe51c005f90 comp rx=0 tx=0).stop 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 msgr2=0x7fe528197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 0x7fe528197620 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e9f0 tx=0x7fe51800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 shutdown_connections 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe528102a20 0x7fe52819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe528102060 0x7fe528197620 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fe5100a3a60 0x7fe5100a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 --2- 172.21.15.38:0/3364778196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe528100e80 0x7fe5281970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 >> 172.21.15.38:0/3364778196 conn(0x7fe5280fc460 msgr2=0x7fe528105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 shutdown_connections 2024-04-25T13:04:50.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:04:50.840+0000 7fe52dd44640 1 -- 172.21.15.38:0/3364778196 wait complete. 2024-04-25T13:04:50.850 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T13:04:50.876 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T13:04:50.886 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:04:50.919 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:50.919 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:04:50.921 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T13:04:50.971 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:04:51.199 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d12c9abd447b4b4ba48d869e564148b3 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:04:51.201 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T13:04:51.211 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T13:04:51.295 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:51.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:50 smithi104 ceph-mon[32726]: mgrmap e22: x(active, since 1.0236s), standbys: z, y 2024-04-25T13:04:51.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:50 smithi104 ceph-mon[32726]: [25/Apr/2024:13:04:49] ENGINE Bus STARTING 2024-04-25T13:04:51.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:50 smithi104 ceph-mon[32726]: [25/Apr/2024:13:04:49] ENGINE Serving on https://172.21.15.38:7150 2024-04-25T13:04:51.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:50 smithi104 ceph-mon[32726]: [25/Apr/2024:13:04:49] ENGINE Serving on http://172.21.15.38:8765 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:50 smithi104 ceph-mon[32726]: [25/Apr/2024:13:04:49] ENGINE Bus STARTED 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:50 smithi104 ceph-mon[32726]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:50 smithi178 ceph-mon[33233]: mgrmap e22: x(active, since 1.0236s), standbys: z, y 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:50 smithi178 ceph-mon[33233]: [25/Apr/2024:13:04:49] ENGINE Bus STARTING 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:50 smithi178 ceph-mon[33233]: [25/Apr/2024:13:04:49] ENGINE Serving on https://172.21.15.38:7150 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:50 smithi178 ceph-mon[33233]: [25/Apr/2024:13:04:49] ENGINE Serving on http://172.21.15.38:8765 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:50 smithi178 ceph-mon[33233]: [25/Apr/2024:13:04:49] ENGINE Bus STARTED 2024-04-25T13:04:51.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:50 smithi178 ceph-mon[33233]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:51.309 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T13:04:51.345 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T13:04:51.345 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:04:51.346 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:04:51.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T13:04:51.375 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:51.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:51.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:51.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:51 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:04:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:51 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.073 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:04:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:52.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:51 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Unable to set osd_memory_target on smithi178 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Unable to set osd_memory_target on smithi104 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Unable to set osd_memory_target on smithi038 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:53.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:53.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T13:04:53.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T13:04:53.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:52 smithi178 ceph-mon[33233]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T13:04:53.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Unable to set osd_memory_target on smithi178 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Unable to set osd_memory_target on smithi104 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T13:04:53.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Unable to set osd_memory_target on smithi038 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:53.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:53.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T13:04:53.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T13:04:53.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:52 smithi038 ceph-mon[29400]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T13:04:53.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Unable to set osd_memory_target on smithi178 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Unable to set osd_memory_target on smithi104 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Unable to set osd_memory_target on smithi038 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-25T13:04:53.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-25T13:04:53.329 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:52 smithi104 ceph-mon[32726]: Updating smithi178:/etc/ceph/ceph.conf 2024-04-25T13:04:54.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:04:54.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T13:04:54.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:54.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:54.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T13:04:54.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:54.329 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:54.329 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:54.329 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: Updating smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.492 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:54.492 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:04:54.492 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:04:54.492 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:04:54.492 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:04:56.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:56 smithi038 ceph-mon[29400]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-25T13:04:56.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:56 smithi104 ceph-mon[32726]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-25T13:04:56.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:56 smithi178 ceph-mon[33233]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-25T13:04:58.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:04:58 smithi038 ceph-mon[29400]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-04-25T13:04:58.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:04:58 smithi104 ceph-mon[32726]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-04-25T13:04:58.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:04:58 smithi178 ceph-mon[33233]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-04-25T13:05:00.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:00 smithi038 ceph-mon[29400]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-04-25T13:05:00.322 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:05:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:05:00.091+0000 7f9da311f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:05:00.322 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:05:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:05:00.091+0000 7f9da311f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:05:00.322 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:05:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:05:00.091+0000 7f9da311f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:05:00.322 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:05:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:05:00.091+0000 7f9da311f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:05:00.322 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:05:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:05:00.091+0000 7f9da311f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:05:00.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:00 smithi104 ceph-mon[32726]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-04-25T13:05:00.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:00 smithi178 ceph-mon[33233]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-04-25T13:05:02.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:02 smithi104 ceph-mon[32726]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T13:05:02.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:02 smithi104 ceph-mon[32726]: mgrmap e25: x(active, since 12s), standbys: z, y 2024-04-25T13:05:02.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:02 smithi178 ceph-mon[33233]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T13:05:02.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:02 smithi178 ceph-mon[33233]: mgrmap e25: x(active, since 12s), standbys: z, y 2024-04-25T13:05:02.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:02 smithi038 ceph-mon[29400]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T13:05:02.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:02 smithi038 ceph-mon[29400]: mgrmap e25: x(active, since 12s), standbys: z, y 2024-04-25T13:05:04.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:04 smithi178 ceph-mon[33233]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T13:05:04.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:04 smithi038 ceph-mon[29400]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T13:05:04.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:04 smithi104 ceph-mon[32726]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T13:05:06.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:06 smithi178 ceph-mon[33233]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 98 KiB/s wr, 66 op/s 2024-04-25T13:05:06.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:06 smithi104 ceph-mon[32726]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 98 KiB/s wr, 66 op/s 2024-04-25T13:05:06.575 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:06 smithi038 ceph-mon[29400]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 98 KiB/s wr, 66 op/s 2024-04-25T13:05:08.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:08 smithi178 ceph-mon[33233]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 61 KiB/s wr, 41 op/s 2024-04-25T13:05:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:08 smithi038 ceph-mon[29400]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 61 KiB/s wr, 41 op/s 2024-04-25T13:05:08.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:08 smithi104 ceph-mon[32726]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 61 KiB/s wr, 41 op/s 2024-04-25T13:05:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:10 smithi178 ceph-mon[33233]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:05:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:10 smithi038 ceph-mon[29400]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:05:10.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:10 smithi104 ceph-mon[32726]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:05:12.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:12 smithi178 ceph-mon[33233]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:05:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:12 smithi038 ceph-mon[29400]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:05:12.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:12 smithi104 ceph-mon[32726]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:05:14.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:14 smithi178 ceph-mon[33233]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:05:14.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:14 smithi038 ceph-mon[29400]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:05:14.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:14 smithi104 ceph-mon[32726]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:05:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:16 smithi178 ceph-mon[33233]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-25T13:05:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:16 smithi038 ceph-mon[29400]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-25T13:05:16.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:16 smithi104 ceph-mon[32726]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-25T13:05:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:18 smithi178 ceph-mon[33233]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T13:05:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:18 smithi038 ceph-mon[29400]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T13:05:18.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:18 smithi104 ceph-mon[32726]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T13:05:20.376 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:20 smithi104 ceph-mon[32726]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:05:20.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:20 smithi178 ceph-mon[33233]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:05:20.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:20 smithi038 ceph-mon[29400]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:05:22.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:22 smithi178 ceph-mon[33233]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T13:05:22.570 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:22 smithi104 ceph-mon[32726]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T13:05:22.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:22 smithi038 ceph-mon[29400]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T13:05:24.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:24 smithi178 ceph-mon[33233]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:24.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:24 smithi038 ceph-mon[29400]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:24.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:24 smithi104 ceph-mon[32726]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:26.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:26 smithi178 ceph-mon[33233]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:05:26.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:26 smithi038 ceph-mon[29400]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:05:26.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:26 smithi104 ceph-mon[32726]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:05:28.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:28 smithi178 ceph-mon[33233]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T13:05:28.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:28 smithi038 ceph-mon[29400]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T13:05:28.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:28 smithi104 ceph-mon[32726]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T13:05:30.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:30 smithi178 ceph-mon[33233]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:30.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:30 smithi038 ceph-mon[29400]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:30.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:30 smithi104 ceph-mon[32726]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:32.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:32 smithi104 ceph-mon[32726]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:32.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:32 smithi178 ceph-mon[33233]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:32.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:32 smithi038 ceph-mon[29400]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:34.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:34 smithi178 ceph-mon[33233]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:34.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:34 smithi038 ceph-mon[29400]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:34.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:34 smithi104 ceph-mon[32726]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:36 smithi178 ceph-mon[33233]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:36 smithi038 ceph-mon[29400]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:36.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:36 smithi104 ceph-mon[32726]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:38 smithi178 ceph-mon[33233]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:05:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:38 smithi038 ceph-mon[29400]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:05:38.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:38 smithi104 ceph-mon[32726]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:05:40.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:40 smithi178 ceph-mon[33233]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:40.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:40 smithi038 ceph-mon[29400]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:40.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:40 smithi104 ceph-mon[32726]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:42.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:42 smithi178 ceph-mon[33233]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:42.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:42 smithi038 ceph-mon[29400]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:42.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:42 smithi104 ceph-mon[32726]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:44.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:44 smithi178 ceph-mon[33233]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:44.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:44 smithi038 ceph-mon[29400]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:44 smithi104 ceph-mon[32726]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:46.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:46 smithi178 ceph-mon[33233]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:46 smithi038 ceph-mon[29400]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:46.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:46 smithi104 ceph-mon[32726]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:48.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:48 smithi178 ceph-mon[33233]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:48.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:48 smithi038 ceph-mon[29400]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:48 smithi104 ceph-mon[32726]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:50.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:50 smithi178 ceph-mon[33233]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:50.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:50 smithi038 ceph-mon[29400]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:50.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:50 smithi104 ceph-mon[32726]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:52.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:52 smithi178 ceph-mon[33233]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:52.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:52 smithi038 ceph-mon[29400]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:52.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:52 smithi104 ceph-mon[32726]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:54.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:54 smithi178 ceph-mon[33233]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:54.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:54 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:54.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:54 smithi038 ceph-mon[29400]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:54.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:54.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:54.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:54 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:54.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:54 smithi104 ceph-mon[32726]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:54.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:05:54.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:54.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:54 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:55.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:55.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:55.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:55.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:55.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:55.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:55.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:05:55.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:05:55.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:05:56.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:56 smithi178 ceph-mon[33233]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:56.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:56 smithi038 ceph-mon[29400]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:56.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:56 smithi104 ceph-mon[32726]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:05:58.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:05:58 smithi178 ceph-mon[33233]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:58.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:05:58 smithi038 ceph-mon[29400]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:05:58.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:05:58 smithi104 ceph-mon[32726]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:00.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:00 smithi038 ceph-mon[29400]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:00 smithi104 ceph-mon[32726]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:00.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:00 smithi178 ceph-mon[33233]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:02.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:02 smithi038 ceph-mon[29400]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:02.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:02 smithi104 ceph-mon[32726]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:02.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:02 smithi178 ceph-mon[33233]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:04.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:04 smithi038 ceph-mon[29400]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:04.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:04 smithi104 ceph-mon[32726]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:04.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:04 smithi178 ceph-mon[33233]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:06.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:06 smithi038 ceph-mon[29400]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:06.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:06 smithi104 ceph-mon[32726]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:06.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:06 smithi178 ceph-mon[33233]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:08.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:08 smithi038 ceph-mon[29400]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:08.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:08 smithi104 ceph-mon[32726]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:08.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:08 smithi178 ceph-mon[33233]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:10 smithi038 ceph-mon[29400]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:10.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:10 smithi104 ceph-mon[32726]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:10 smithi178 ceph-mon[33233]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:12 smithi038 ceph-mon[29400]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:12 smithi104 ceph-mon[32726]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:12.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:12 smithi178 ceph-mon[33233]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:14.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:14 smithi038 ceph-mon[29400]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:14.573 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:14 smithi104 ceph-mon[32726]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:14.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:14 smithi178 ceph-mon[33233]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:06:15.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:15 smithi178 ceph-mon[33233]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:15.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:15 smithi038 ceph-mon[29400]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:15.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:15 smithi104 ceph-mon[32726]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:06:17.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:17 smithi178 ceph-mon[33233]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:06:18.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:17 smithi038 ceph-mon[29400]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:06:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:17 smithi104 ceph-mon[32726]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:06:19.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:19 smithi178 ceph-mon[33233]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:06:20.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:19 smithi038 ceph-mon[29400]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:06:20.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:19 smithi104 ceph-mon[32726]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:06:21.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:21 smithi178 ceph-mon[33233]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-25T13:06:21.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:21 smithi178 ceph-mon[33233]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T13:06:22.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:21 smithi038 ceph-mon[29400]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-25T13:06:22.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:21 smithi038 ceph-mon[29400]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T13:06:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:21 smithi104 ceph-mon[32726]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-25T13:06:22.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:21 smithi104 ceph-mon[32726]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T13:06:23.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:23 smithi178 ceph-mon[33233]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:24.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:23 smithi038 ceph-mon[29400]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:24.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:23 smithi104 ceph-mon[32726]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:25.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:25 smithi178 ceph-mon[33233]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T13:06:26.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:25 smithi104 ceph-mon[32726]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T13:06:26.074 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:25 smithi038 ceph-mon[29400]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T13:06:27.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:27 smithi178 ceph-mon[33233]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:28.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:27 smithi038 ceph-mon[29400]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:28.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:27 smithi104 ceph-mon[32726]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:29.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:29 smithi178 ceph-mon[33233]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:30.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:29 smithi038 ceph-mon[29400]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:30.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:29 smithi104 ceph-mon[32726]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T13:06:31.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:31 smithi178 ceph-mon[33233]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T13:06:32.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:31 smithi038 ceph-mon[29400]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T13:06:32.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:31 smithi104 ceph-mon[32726]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T13:06:33.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:33 smithi178 ceph-mon[33233]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:34.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:33 smithi038 ceph-mon[29400]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:34.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:33 smithi104 ceph-mon[32726]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:35.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:35 smithi178 ceph-mon[33233]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:36.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:35 smithi038 ceph-mon[29400]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:36.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:35 smithi104 ceph-mon[32726]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:37.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:37 smithi178 ceph-mon[33233]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:38.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:37 smithi038 ceph-mon[29400]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:38.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:37 smithi104 ceph-mon[32726]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:40.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:39 smithi038 ceph-mon[29400]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:40.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:39 smithi104 ceph-mon[32726]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:40.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:39 smithi178 ceph-mon[33233]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:42.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:41 smithi038 ceph-mon[29400]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:42.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:41 smithi104 ceph-mon[32726]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:42.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:41 smithi178 ceph-mon[33233]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:44.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:43 smithi038 ceph-mon[29400]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:44.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:43 smithi104 ceph-mon[32726]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:44.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:43 smithi178 ceph-mon[33233]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:46.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:45 smithi038 ceph-mon[29400]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:46.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:45 smithi104 ceph-mon[32726]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:46.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:45 smithi178 ceph-mon[33233]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:48.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:47 smithi038 ceph-mon[29400]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:48.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:47 smithi104 ceph-mon[32726]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:48.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:47 smithi178 ceph-mon[33233]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:50.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:49 smithi038 ceph-mon[29400]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:50.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:49 smithi104 ceph-mon[32726]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:50.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:49 smithi178 ceph-mon[33233]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:52.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:51 smithi178 ceph-mon[33233]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:52.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:51 smithi038 ceph-mon[29400]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:51 smithi104 ceph-mon[32726]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:54.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:53 smithi038 ceph-mon[29400]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:54.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:53 smithi104 ceph-mon[32726]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:54.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:53 smithi178 ceph-mon[33233]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:55 smithi038 ceph-mon[29400]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:55 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:55 smithi104 ceph-mon[32726]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:56.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:56.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:55 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:55 smithi178 ceph-mon[33233]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:06:56.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:06:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:55 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:57.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:56 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:57.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:56 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:57.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:56 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:57.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:56 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:57.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:56 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:57.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:56 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:57.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:56 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:06:57.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:56 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:06:57.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:56 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:06:58.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:06:58 smithi038 ceph-mon[29400]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:58.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:06:58 smithi104 ceph-mon[32726]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:06:58.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:06:58 smithi178 ceph-mon[33233]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:00.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:00 smithi038 ceph-mon[29400]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:00.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:00 smithi104 ceph-mon[32726]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:00.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:00 smithi178 ceph-mon[33233]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:02.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:02 smithi038 ceph-mon[29400]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:02.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:02 smithi104 ceph-mon[32726]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:02.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:02 smithi178 ceph-mon[33233]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:04.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:04 smithi038 ceph-mon[29400]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:04 smithi104 ceph-mon[32726]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:04.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:04 smithi178 ceph-mon[33233]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:06.321 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:06 smithi038 ceph-mon[29400]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:06.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:06 smithi104 ceph-mon[32726]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:06.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:06 smithi178 ceph-mon[33233]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:08.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:08 smithi038 ceph-mon[29400]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:08.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:08 smithi104 ceph-mon[32726]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:08.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:08 smithi178 ceph-mon[33233]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:10.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:10 smithi178 ceph-mon[33233]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:10.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:10 smithi038 ceph-mon[29400]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:10.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:10 smithi104 ceph-mon[32726]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:12.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:12 smithi178 ceph-mon[33233]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:12.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:12 smithi038 ceph-mon[29400]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:12 smithi104 ceph-mon[32726]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:14.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:14 smithi178 ceph-mon[33233]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:14.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:14 smithi038 ceph-mon[29400]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:14.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:14 smithi104 ceph-mon[32726]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:07:16.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:16 smithi178 ceph-mon[33233]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:16.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:16 smithi038 ceph-mon[29400]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:16 smithi104 ceph-mon[32726]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:07:18.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:18 smithi178 ceph-mon[33233]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:07:18.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:18 smithi038 ceph-mon[29400]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:07:18.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:18 smithi104 ceph-mon[32726]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:07:20.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:20 smithi178 ceph-mon[33233]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:07:20.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:20 smithi038 ceph-mon[29400]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:07:20.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:20 smithi104 ceph-mon[32726]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:07:22.376 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:22 smithi104 ceph-mon[32726]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:07:22.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:22 smithi178 ceph-mon[33233]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:07:22.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:22 smithi038 ceph-mon[29400]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:07:24.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:24 smithi178 ceph-mon[33233]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:24.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:24 smithi038 ceph-mon[29400]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:24.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:24 smithi104 ceph-mon[32726]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:26.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:26 smithi178 ceph-mon[33233]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:26.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:26 smithi038 ceph-mon[29400]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:26.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:26 smithi104 ceph-mon[32726]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:28.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:28 smithi178 ceph-mon[33233]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:28.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:28 smithi038 ceph-mon[29400]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:28.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:28 smithi104 ceph-mon[32726]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:30.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:30 smithi178 ceph-mon[33233]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:30.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:30 smithi038 ceph-mon[29400]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:30.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:30 smithi104 ceph-mon[32726]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:32.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:32 smithi178 ceph-mon[33233]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:32.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:32 smithi038 ceph-mon[29400]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:32.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:32 smithi104 ceph-mon[32726]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:34.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:34 smithi178 ceph-mon[33233]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:34.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:34 smithi038 ceph-mon[29400]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:34.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:34 smithi104 ceph-mon[32726]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:36.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:36 smithi178 ceph-mon[33233]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:36.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:36 smithi038 ceph-mon[29400]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:36 smithi104 ceph-mon[32726]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:38.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:38 smithi178 ceph-mon[33233]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:38 smithi038 ceph-mon[29400]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:38.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:38 smithi104 ceph-mon[32726]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:40.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:40 smithi178 ceph-mon[33233]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:40.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:40 smithi038 ceph-mon[29400]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:40.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:40 smithi104 ceph-mon[32726]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:42.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:42 smithi178 ceph-mon[33233]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:42.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:42 smithi038 ceph-mon[29400]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:42.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:42 smithi104 ceph-mon[32726]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:44.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:44 smithi178 ceph-mon[33233]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:44.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:44 smithi038 ceph-mon[29400]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:44 smithi104 ceph-mon[32726]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:46.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:46 smithi178 ceph-mon[33233]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:46.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:46 smithi038 ceph-mon[29400]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:46.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:46 smithi104 ceph-mon[32726]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:48.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:48 smithi178 ceph-mon[33233]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:48.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:48 smithi038 ceph-mon[29400]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:48 smithi104 ceph-mon[32726]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:50.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:50 smithi178 ceph-mon[33233]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:50.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:50 smithi038 ceph-mon[29400]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:50.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:50 smithi104 ceph-mon[32726]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:52.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:52 smithi178 ceph-mon[33233]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:52.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:52 smithi038 ceph-mon[29400]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:52.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:52 smithi104 ceph-mon[32726]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:54.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:54 smithi178 ceph-mon[33233]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:54.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:54 smithi038 ceph-mon[29400]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:54.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:54 smithi104 ceph-mon[32726]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:56.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:56 smithi178 ceph-mon[33233]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:56.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:56 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:56.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:56 smithi038 ceph-mon[29400]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:56.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:56 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:56.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:56 smithi104 ceph-mon[32726]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:07:56.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:56 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:07:57.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:57.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:57.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:07:57.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:57.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:57.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:07:57.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:07:57.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:07:57.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:07:58.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:07:58 smithi178 ceph-mon[33233]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:58.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:07:58 smithi038 ceph-mon[29400]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:07:58.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:07:58 smithi104 ceph-mon[32726]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:00.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:00 smithi178 ceph-mon[33233]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:00.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:00 smithi038 ceph-mon[29400]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:00 smithi104 ceph-mon[32726]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:02.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:02 smithi178 ceph-mon[33233]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:02.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:02 smithi038 ceph-mon[29400]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:02.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:02 smithi104 ceph-mon[32726]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:04.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:04 smithi178 ceph-mon[33233]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:04.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:04 smithi038 ceph-mon[29400]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:04.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:04 smithi104 ceph-mon[32726]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:06.489 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:06 smithi178 ceph-mon[33233]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:06.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:06 smithi038 ceph-mon[29400]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:06.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:06 smithi104 ceph-mon[32726]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:08.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:08 smithi038 ceph-mon[29400]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:08.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:08 smithi104 ceph-mon[32726]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:08.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:08 smithi178 ceph-mon[33233]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:10.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:10 smithi038 ceph-mon[29400]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:10.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:10 smithi104 ceph-mon[32726]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:10.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:10 smithi178 ceph-mon[33233]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:12.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:12 smithi038 ceph-mon[29400]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:12 smithi104 ceph-mon[32726]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:12.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:12 smithi178 ceph-mon[33233]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:14.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:14 smithi038 ceph-mon[29400]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:14.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:14 smithi104 ceph-mon[32726]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:14.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:14 smithi178 ceph-mon[33233]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:08:16.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:16 smithi038 ceph-mon[29400]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:16 smithi104 ceph-mon[32726]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:16.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:16 smithi178 ceph-mon[33233]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:08:18.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:18 smithi038 ceph-mon[29400]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:08:18.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:18 smithi104 ceph-mon[32726]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:08:18.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:18 smithi178 ceph-mon[33233]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:08:20.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:20 smithi038 ceph-mon[29400]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:08:20.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:20 smithi104 ceph-mon[32726]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:08:20.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:20 smithi178 ceph-mon[33233]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:08:22.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:22 smithi038 ceph-mon[29400]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:08:22.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:22 smithi104 ceph-mon[32726]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:08:22.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:22 smithi178 ceph-mon[33233]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:08:24.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:24 smithi038 ceph-mon[29400]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:24.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:24 smithi104 ceph-mon[32726]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:24.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:24 smithi178 ceph-mon[33233]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:26.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:26 smithi038 ceph-mon[29400]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:26.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:26 smithi104 ceph-mon[32726]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:26.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:26 smithi178 ceph-mon[33233]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:28.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:28 smithi038 ceph-mon[29400]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:28.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:28 smithi104 ceph-mon[32726]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:28.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:28 smithi178 ceph-mon[33233]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:30.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:30 smithi038 ceph-mon[29400]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:30.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:30 smithi104 ceph-mon[32726]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:30.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:30 smithi178 ceph-mon[33233]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:32.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:32 smithi104 ceph-mon[32726]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:32.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:32 smithi178 ceph-mon[33233]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:32.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:32 smithi038 ceph-mon[29400]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:34.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:34 smithi178 ceph-mon[33233]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:34.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:34 smithi038 ceph-mon[29400]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:34.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:34 smithi104 ceph-mon[32726]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:35.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:35 smithi178 ceph-mon[33233]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:35.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:35 smithi038 ceph-mon[29400]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:35.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:35 smithi104 ceph-mon[32726]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:37.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:37 smithi178 ceph-mon[33233]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:38.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:37 smithi038 ceph-mon[29400]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:38.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:37 smithi104 ceph-mon[32726]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:39.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:39 smithi178 ceph-mon[33233]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:40.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:39 smithi038 ceph-mon[29400]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:40.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:39 smithi104 ceph-mon[32726]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:41.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:41 smithi178 ceph-mon[33233]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:42.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:41 smithi038 ceph-mon[29400]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:42.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:41 smithi104 ceph-mon[32726]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:43.989 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:43 smithi178 ceph-mon[33233]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:44.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:43 smithi038 ceph-mon[29400]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:44.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:43 smithi104 ceph-mon[32726]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:45.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:45 smithi178 ceph-mon[33233]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:46.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:45 smithi038 ceph-mon[29400]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:46.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:45 smithi104 ceph-mon[32726]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:47.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:47 smithi178 ceph-mon[33233]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:48.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:47 smithi038 ceph-mon[29400]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:48.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:47 smithi104 ceph-mon[32726]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:49.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:49 smithi178 ceph-mon[33233]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:50.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:49 smithi038 ceph-mon[29400]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:50.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:49 smithi104 ceph-mon[32726]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:52.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:51 smithi038 ceph-mon[29400]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:52.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:51 smithi104 ceph-mon[32726]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:52.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:51 smithi178 ceph-mon[33233]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:54.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:53 smithi178 ceph-mon[33233]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:54.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:53 smithi038 ceph-mon[29400]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:54.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:53 smithi104 ceph-mon[32726]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:56.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:55 smithi038 ceph-mon[29400]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:56.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:55 smithi104 ceph-mon[32726]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:55 smithi178 ceph-mon[33233]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:08:58.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:57 smithi038 ceph-mon[29400]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:58.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:58.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:58.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:58.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:08:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:57 smithi104 ceph-mon[32726]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:58.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:58.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:58.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:58.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:08:58.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:57 smithi178 ceph-mon[33233]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:08:58.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:08:58.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:08:58.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:08:58.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:09:00.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:08:59 smithi038 ceph-mon[29400]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:00.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:08:59 smithi104 ceph-mon[32726]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:00.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:08:59 smithi178 ceph-mon[33233]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:02.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:01 smithi038 ceph-mon[29400]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:02.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:01 smithi104 ceph-mon[32726]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:02.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:01 smithi178 ceph-mon[33233]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:04.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:03 smithi038 ceph-mon[29400]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:04.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:03 smithi104 ceph-mon[32726]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:03 smithi178 ceph-mon[33233]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:06.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:05 smithi038 ceph-mon[29400]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:06.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:05 smithi104 ceph-mon[32726]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:06.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:05 smithi178 ceph-mon[33233]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:08.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:07 smithi038 ceph-mon[29400]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:08.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:07 smithi104 ceph-mon[32726]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:08.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:07 smithi178 ceph-mon[33233]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:10.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:09 smithi038 ceph-mon[29400]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:10.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:09 smithi104 ceph-mon[32726]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:10.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:09 smithi178 ceph-mon[33233]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:12.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:11 smithi038 ceph-mon[29400]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:12.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:11 smithi104 ceph-mon[32726]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:12.239 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:11 smithi178 ceph-mon[33233]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:14.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:13 smithi038 ceph-mon[29400]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:14.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:13 smithi104 ceph-mon[32726]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:14.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:13 smithi178 ceph-mon[33233]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:09:16.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:15 smithi038 ceph-mon[29400]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:16.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:15 smithi104 ceph-mon[32726]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:16.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:15 smithi178 ceph-mon[33233]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:09:18.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:17 smithi038 ceph-mon[29400]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:09:18.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:17 smithi104 ceph-mon[32726]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:09:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:17 smithi178 ceph-mon[33233]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:09:20.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:19 smithi104 ceph-mon[32726]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:09:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:19 smithi178 ceph-mon[33233]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:09:20.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:19 smithi038 ceph-mon[29400]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:09:22.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:21 smithi178 ceph-mon[33233]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:09:22.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:21 smithi038 ceph-mon[29400]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:09:22.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:21 smithi104 ceph-mon[32726]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:09:24.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:23 smithi104 ceph-mon[32726]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:24.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:23 smithi178 ceph-mon[33233]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:24.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:23 smithi038 ceph-mon[29400]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:26.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:25 smithi178 ceph-mon[33233]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:26.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:25 smithi038 ceph-mon[29400]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:26.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:25 smithi104 ceph-mon[32726]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:28.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:27 smithi178 ceph-mon[33233]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:28.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:27 smithi038 ceph-mon[29400]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:28.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:27 smithi104 ceph-mon[32726]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:30.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:29 smithi178 ceph-mon[33233]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:30.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:29 smithi038 ceph-mon[29400]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:30.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:29 smithi104 ceph-mon[32726]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:32.124 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:31 smithi038 ceph-mon[29400]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:32.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:31 smithi178 ceph-mon[33233]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:32.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:31 smithi104 ceph-mon[32726]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:33 smithi178 ceph-mon[33233]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:33 smithi038 ceph-mon[29400]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:33 smithi104 ceph-mon[32726]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:36.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:35 smithi178 ceph-mon[33233]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:36.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:35 smithi038 ceph-mon[29400]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:35 smithi104 ceph-mon[32726]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:38.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:37 smithi178 ceph-mon[33233]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:38.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:37 smithi038 ceph-mon[29400]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:38.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:37 smithi104 ceph-mon[32726]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:40.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:39 smithi178 ceph-mon[33233]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:39 smithi038 ceph-mon[29400]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:40.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:39 smithi104 ceph-mon[32726]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:42.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:41 smithi178 ceph-mon[33233]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:42.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:41 smithi038 ceph-mon[29400]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:41 smithi104 ceph-mon[32726]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:44.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:43 smithi178 ceph-mon[33233]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:44.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:43 smithi038 ceph-mon[29400]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:44.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:43 smithi104 ceph-mon[32726]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:46.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:45 smithi178 ceph-mon[33233]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:46.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:45 smithi038 ceph-mon[29400]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:45 smithi104 ceph-mon[32726]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:48.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:47 smithi178 ceph-mon[33233]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:48.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:47 smithi038 ceph-mon[29400]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:48.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:47 smithi104 ceph-mon[32726]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:50.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:49 smithi178 ceph-mon[33233]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:50.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:49 smithi038 ceph-mon[29400]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:49 smithi104 ceph-mon[32726]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:52.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:51 smithi178 ceph-mon[33233]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:52.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:51 smithi038 ceph-mon[29400]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:52.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:51 smithi104 ceph-mon[32726]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:54.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:53 smithi178 ceph-mon[33233]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:53 smithi038 ceph-mon[29400]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:54.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:53 smithi104 ceph-mon[32726]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:55 smithi178 ceph-mon[33233]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:56.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:55 smithi038 ceph-mon[29400]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:56.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:55 smithi104 ceph-mon[32726]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:09:58.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:57 smithi178 ceph-mon[33233]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:58.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:57 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:58.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:57 smithi038 ceph-mon[29400]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:58.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:57 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:58.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:57 smithi104 ceph-mon[32726]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:09:58.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:57 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:09:59.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:58 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:59.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:58 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:59.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:58 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:09:59.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:58 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:59.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:58 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:59.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:58 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:09:59.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:58 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:59.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:58 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:09:59.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:58 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:10:00.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:09:59 smithi178 ceph-mon[33233]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:00.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:09:59 smithi038 ceph-mon[29400]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:00.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:09:59 smithi104 ceph-mon[32726]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:01.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:00 smithi178 ceph-mon[33233]: overall HEALTH_OK 2024-04-25T13:10:01.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:00 smithi038 ceph-mon[29400]: overall HEALTH_OK 2024-04-25T13:10:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:00 smithi104 ceph-mon[32726]: overall HEALTH_OK 2024-04-25T13:10:02.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:01 smithi038 ceph-mon[29400]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:02.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:01 smithi178 ceph-mon[33233]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:02.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:01 smithi104 ceph-mon[32726]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:03 smithi178 ceph-mon[33233]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:04.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:03 smithi038 ceph-mon[29400]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:03 smithi104 ceph-mon[32726]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:06.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:05 smithi178 ceph-mon[33233]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:06.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:05 smithi038 ceph-mon[29400]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:06.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:05 smithi104 ceph-mon[32726]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:08.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:07 smithi178 ceph-mon[33233]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:08.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:07 smithi038 ceph-mon[29400]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:08.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:07 smithi104 ceph-mon[32726]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:10.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:09 smithi178 ceph-mon[33233]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:10.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:09 smithi038 ceph-mon[29400]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:10.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:09 smithi104 ceph-mon[32726]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:12.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:11 smithi178 ceph-mon[33233]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:12.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:11 smithi038 ceph-mon[29400]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:12.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:11 smithi104 ceph-mon[32726]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:14.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:13 smithi178 ceph-mon[33233]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:14.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:13 smithi038 ceph-mon[29400]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:14.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:13 smithi104 ceph-mon[32726]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T13:10:16.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:15 smithi178 ceph-mon[33233]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:15 smithi038 ceph-mon[29400]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:16.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:15 smithi104 ceph-mon[32726]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T13:10:18.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:18 smithi038 ceph-mon[29400]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:10:18.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:18 smithi104 ceph-mon[32726]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:10:18.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:18 smithi178 ceph-mon[33233]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:10:20.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:20 smithi038 ceph-mon[29400]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:10:20.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:20 smithi104 ceph-mon[32726]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:10:20.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:20 smithi178 ceph-mon[33233]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:10:22.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:22 smithi038 ceph-mon[29400]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:10:22.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:22 smithi104 ceph-mon[32726]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:10:22.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:22 smithi178 ceph-mon[33233]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:10:24.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:24 smithi178 ceph-mon[33233]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:24.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:24 smithi038 ceph-mon[29400]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:24.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:24 smithi104 ceph-mon[32726]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:26.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:26 smithi038 ceph-mon[29400]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:26.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:26 smithi104 ceph-mon[32726]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:26.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:26 smithi178 ceph-mon[33233]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:28.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:28 smithi038 ceph-mon[29400]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:28.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:28 smithi104 ceph-mon[32726]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:28.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:28 smithi178 ceph-mon[33233]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:30.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:30 smithi038 ceph-mon[29400]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:30.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:30 smithi104 ceph-mon[32726]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:30.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:30 smithi178 ceph-mon[33233]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:32.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:32 smithi038 ceph-mon[29400]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:32.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:32 smithi104 ceph-mon[32726]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:32.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:32 smithi178 ceph-mon[33233]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:34 smithi104 ceph-mon[32726]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:34.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:34 smithi178 ceph-mon[33233]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:34.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:34 smithi038 ceph-mon[29400]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:36.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:36 smithi178 ceph-mon[33233]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:36.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:36 smithi038 ceph-mon[29400]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:36 smithi104 ceph-mon[32726]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:38.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:38 smithi178 ceph-mon[33233]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:38.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:38 smithi038 ceph-mon[29400]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:38.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:38 smithi104 ceph-mon[32726]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:40.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:40 smithi178 ceph-mon[33233]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:40.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:40 smithi038 ceph-mon[29400]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:40.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:40 smithi104 ceph-mon[32726]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:42.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:42 smithi178 ceph-mon[33233]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:42.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:42 smithi038 ceph-mon[29400]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:42.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:42 smithi104 ceph-mon[32726]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:44.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:44 smithi178 ceph-mon[33233]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:44.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:44 smithi038 ceph-mon[29400]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:44 smithi104 ceph-mon[32726]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:46.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:46 smithi178 ceph-mon[33233]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:46.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:46 smithi038 ceph-mon[29400]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:46.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:46 smithi104 ceph-mon[32726]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:48.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:48 smithi178 ceph-mon[33233]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:48.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:48 smithi038 ceph-mon[29400]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:48 smithi104 ceph-mon[32726]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:50.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:50 smithi178 ceph-mon[33233]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:50.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:50 smithi038 ceph-mon[29400]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:50.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:50 smithi104 ceph-mon[32726]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:52.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:52 smithi178 ceph-mon[33233]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:52.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:52 smithi038 ceph-mon[29400]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:52.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:52 smithi104 ceph-mon[32726]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:54 smithi178 ceph-mon[33233]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:54.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:54 smithi038 ceph-mon[29400]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:54.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:54 smithi104 ceph-mon[32726]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:56.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:56 smithi178 ceph-mon[33233]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:56.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:56 smithi038 ceph-mon[29400]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:56.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:56 smithi104 ceph-mon[32726]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:10:58.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:58 smithi178 ceph-mon[33233]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:58.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:58 smithi038 ceph-mon[29400]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:58.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:58 smithi104 ceph-mon[32726]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:10:59.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:10:59 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:10:59 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:59.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:10:59 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:00.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:00 smithi178 ceph-mon[33233]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:00.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:00 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:00.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:00 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:00.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:00 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:11:00.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:00 smithi038 ceph-mon[29400]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:00.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:00 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:00.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:00 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:00.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:00 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:11:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:00 smithi104 ceph-mon[32726]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:00 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:00 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:00 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:11:02.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:02 smithi178 ceph-mon[33233]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:02.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:02 smithi038 ceph-mon[29400]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:02.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:02 smithi104 ceph-mon[32726]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:04.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:04 smithi178 ceph-mon[33233]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:04.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:04 smithi038 ceph-mon[29400]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:04.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:04 smithi104 ceph-mon[32726]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:06.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:06 smithi178 ceph-mon[33233]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:11:06.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:06 smithi038 ceph-mon[29400]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:11:06.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:06 smithi104 ceph-mon[32726]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:11:07.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:07 smithi178 ceph-mon[33233]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:11:07.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:07 smithi038 ceph-mon[29400]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:11:07.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:07 smithi104 ceph-mon[32726]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:11:08.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:08 smithi178 ceph-mon[33233]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:08.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:08 smithi038 ceph-mon[29400]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:08.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:08 smithi104 ceph-mon[32726]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:10.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:10 smithi178 ceph-mon[33233]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:10.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:10 smithi038 ceph-mon[29400]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:10.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:10 smithi104 ceph-mon[32726]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:12.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:12 smithi178 ceph-mon[33233]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:12.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:12 smithi038 ceph-mon[29400]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:12 smithi104 ceph-mon[32726]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:14.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:14 smithi178 ceph-mon[33233]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:14.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:14 smithi038 ceph-mon[29400]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:14.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:14 smithi104 ceph-mon[32726]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:11:16.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:16 smithi178 ceph-mon[33233]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:16.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:16 smithi178 ceph-mon[33233]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:11:16.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:16 smithi038 ceph-mon[29400]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:16.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:16 smithi038 ceph-mon[29400]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:11:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:16 smithi104 ceph-mon[32726]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:16.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:16 smithi104 ceph-mon[32726]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:11:18.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:18 smithi178 ceph-mon[33233]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:18.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:18 smithi038 ceph-mon[29400]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:18.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:18 smithi104 ceph-mon[32726]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:20.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:20 smithi178 ceph-mon[33233]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:20 smithi038 ceph-mon[29400]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:20.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:20 smithi104 ceph-mon[32726]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:11:22.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:22 smithi178 ceph-mon[33233]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:11:22.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:22 smithi038 ceph-mon[29400]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:11:22.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:22 smithi104 ceph-mon[32726]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:11:24.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:24 smithi178 ceph-mon[33233]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:11:24.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:24 smithi038 ceph-mon[29400]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:11:24.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:24 smithi104 ceph-mon[32726]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:11:26.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:26 smithi038 ceph-mon[29400]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:11:26.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:26 smithi104 ceph-mon[32726]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:11:26.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:26 smithi178 ceph-mon[33233]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:11:28.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:28 smithi038 ceph-mon[29400]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:11:28.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:28 smithi104 ceph-mon[32726]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:11:28.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:28 smithi178 ceph-mon[33233]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:11:30.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:30 smithi038 ceph-mon[29400]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:30.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:30 smithi104 ceph-mon[32726]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:30.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:30 smithi178 ceph-mon[33233]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:32.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:32 smithi038 ceph-mon[29400]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:32.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:32 smithi104 ceph-mon[32726]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:32.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:32 smithi178 ceph-mon[33233]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:34.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:34 smithi038 ceph-mon[29400]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:34.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:34 smithi104 ceph-mon[32726]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:34.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:34 smithi178 ceph-mon[33233]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:36.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:36 smithi038 ceph-mon[29400]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:36.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:36 smithi104 ceph-mon[32726]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:36.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:36 smithi178 ceph-mon[33233]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:38.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:38 smithi038 ceph-mon[29400]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:38.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:38 smithi104 ceph-mon[32726]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:38.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:38 smithi178 ceph-mon[33233]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:40.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:40 smithi038 ceph-mon[29400]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:40.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:40 smithi104 ceph-mon[32726]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:40.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:40 smithi178 ceph-mon[33233]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:42.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:42 smithi038 ceph-mon[29400]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:42.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:42 smithi104 ceph-mon[32726]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:42.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:42 smithi178 ceph-mon[33233]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:44.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:44 smithi038 ceph-mon[29400]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:44 smithi104 ceph-mon[32726]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:44.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:44 smithi178 ceph-mon[33233]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:46.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:46 smithi038 ceph-mon[29400]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:46.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:46 smithi104 ceph-mon[32726]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:46.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:46 smithi178 ceph-mon[33233]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:48.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:48 smithi038 ceph-mon[29400]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:48 smithi104 ceph-mon[32726]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:48.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:48 smithi178 ceph-mon[33233]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:50.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:50 smithi038 ceph-mon[29400]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:50.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:50 smithi104 ceph-mon[32726]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:50 smithi178 ceph-mon[33233]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:52.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:52 smithi038 ceph-mon[29400]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:52.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:52 smithi104 ceph-mon[32726]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:52.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:52 smithi178 ceph-mon[33233]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:54.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:54 smithi038 ceph-mon[29400]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:54.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:54 smithi104 ceph-mon[32726]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:54.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:54 smithi178 ceph-mon[33233]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:56.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:56 smithi178 ceph-mon[33233]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:56.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:56 smithi038 ceph-mon[29400]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:56.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:56 smithi104 ceph-mon[32726]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:11:58.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:11:58 smithi178 ceph-mon[33233]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:58.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:11:58 smithi038 ceph-mon[29400]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:11:58.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:11:58 smithi104 ceph-mon[32726]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:00.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:00 smithi178 ceph-mon[33233]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:00.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:00 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:00.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:00 smithi038 ceph-mon[29400]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:00.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:00 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:00.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:00 smithi104 ceph-mon[32726]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:00.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:00 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:01.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:01 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:01.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:01 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:01.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:01 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:12:01.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:01 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:01.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:01 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:01.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:01 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:12:01.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:01 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:01.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:01 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:01.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:01 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:12:02.723 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:02 smithi038 ceph-mon[29400]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:02.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:02 smithi178 ceph-mon[33233]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:02.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:02 smithi104 ceph-mon[32726]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:04.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:04 smithi178 ceph-mon[33233]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:04.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:04 smithi038 ceph-mon[29400]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:04.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:04 smithi104 ceph-mon[32726]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:06.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:06 smithi178 ceph-mon[33233]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:12:06.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:06 smithi038 ceph-mon[29400]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:12:06.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:06 smithi104 ceph-mon[32726]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:12:08.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:08 smithi178 ceph-mon[33233]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:12:08.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:08 smithi038 ceph-mon[29400]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:12:08.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:08 smithi104 ceph-mon[32726]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:12:10.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:10 smithi178 ceph-mon[33233]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:12:10.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:10 smithi038 ceph-mon[29400]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:12:10.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:10 smithi104 ceph-mon[32726]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:12:11.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:11 smithi178 ceph-mon[33233]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:12:11.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:11 smithi178 ceph-mon[33233]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:12:11.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:11 smithi038 ceph-mon[29400]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:12:11.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:11 smithi038 ceph-mon[29400]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:12:11.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:11 smithi104 ceph-mon[32726]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:12:11.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:11 smithi104 ceph-mon[32726]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:12:14.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:13 smithi038 ceph-mon[29400]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:12:14.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:13 smithi104 ceph-mon[32726]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:12:14.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:13 smithi178 ceph-mon[33233]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:12:16.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:15 smithi038 ceph-mon[29400]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:12:16.074 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:15 smithi104 ceph-mon[32726]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:12:16.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:15 smithi178 ceph-mon[33233]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:12:17.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:16 smithi178 ceph-mon[33233]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:12:17.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:16 smithi038 ceph-mon[29400]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:12:17.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:16 smithi104 ceph-mon[32726]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:12:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:17 smithi178 ceph-mon[33233]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:12:18.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:17 smithi038 ceph-mon[29400]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:12:18.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:17 smithi104 ceph-mon[32726]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:12:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:19 smithi178 ceph-mon[33233]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:12:20.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:19 smithi038 ceph-mon[29400]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:12:20.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:19 smithi104 ceph-mon[32726]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:12:22.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:21 smithi178 ceph-mon[33233]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 328 B/s wr, 11 op/s 2024-04-25T13:12:22.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:21 smithi038 ceph-mon[29400]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 328 B/s wr, 11 op/s 2024-04-25T13:12:22.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:21 smithi104 ceph-mon[32726]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 328 B/s wr, 11 op/s 2024-04-25T13:12:24.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:23 smithi104 ceph-mon[32726]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T13:12:24.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:23 smithi178 ceph-mon[33233]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T13:12:24.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:23 smithi038 ceph-mon[29400]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T13:12:26.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:25 smithi178 ceph-mon[33233]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:12:26.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:25 smithi038 ceph-mon[29400]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:12:26.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:25 smithi104 ceph-mon[32726]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:12:28.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:27 smithi178 ceph-mon[33233]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 187 B/s wr, 9 op/s 2024-04-25T13:12:28.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:27 smithi038 ceph-mon[29400]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 187 B/s wr, 9 op/s 2024-04-25T13:12:28.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:27 smithi104 ceph-mon[32726]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 187 B/s wr, 9 op/s 2024-04-25T13:12:30.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:29 smithi178 ceph-mon[33233]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:12:30.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:29 smithi038 ceph-mon[29400]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:12:30.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:29 smithi104 ceph-mon[32726]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:12:32.125 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:31 smithi038 ceph-mon[29400]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T13:12:32.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:31 smithi178 ceph-mon[33233]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T13:12:32.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:31 smithi104 ceph-mon[32726]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T13:12:34.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:33 smithi178 ceph-mon[33233]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:33 smithi038 ceph-mon[29400]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:33 smithi104 ceph-mon[32726]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:36.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:35 smithi178 ceph-mon[33233]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:36.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:35 smithi038 ceph-mon[29400]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:35 smithi104 ceph-mon[32726]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:38.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:37 smithi178 ceph-mon[33233]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:38.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:37 smithi038 ceph-mon[29400]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:38.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:37 smithi104 ceph-mon[32726]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:40.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:39 smithi178 ceph-mon[33233]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:39 smithi038 ceph-mon[29400]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:40.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:39 smithi104 ceph-mon[32726]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:42.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:41 smithi178 ceph-mon[33233]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:42.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:41 smithi038 ceph-mon[29400]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:41 smithi104 ceph-mon[32726]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:44.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:43 smithi178 ceph-mon[33233]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:44.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:43 smithi038 ceph-mon[29400]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:44.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:43 smithi104 ceph-mon[32726]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T13:12:46.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:45 smithi178 ceph-mon[33233]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:46.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:45 smithi038 ceph-mon[29400]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:45 smithi104 ceph-mon[32726]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:48.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:47 smithi178 ceph-mon[33233]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:48.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:47 smithi038 ceph-mon[29400]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:48.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:47 smithi104 ceph-mon[32726]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:50.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:49 smithi178 ceph-mon[33233]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:50.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:49 smithi038 ceph-mon[29400]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:50.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:49 smithi104 ceph-mon[32726]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:52.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:51 smithi178 ceph-mon[33233]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:52.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:51 smithi038 ceph-mon[29400]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:52.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:51 smithi104 ceph-mon[32726]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:54.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:53 smithi178 ceph-mon[33233]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:54.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:53 smithi038 ceph-mon[29400]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:54.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:53 smithi104 ceph-mon[32726]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:56.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:55 smithi178 ceph-mon[33233]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:56.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:55 smithi038 ceph-mon[29400]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:56.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:55 smithi104 ceph-mon[32726]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:12:58.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:57 smithi178 ceph-mon[33233]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:58.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:57 smithi038 ceph-mon[29400]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:12:58.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:57 smithi104 ceph-mon[32726]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:00.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:12:59 smithi178 ceph-mon[33233]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:00.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:12:59 smithi038 ceph-mon[29400]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:00.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:12:59 smithi104 ceph-mon[32726]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:01.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:00 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:01.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:00 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:01.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:00 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:02.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:01 smithi038 ceph-mon[29400]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:02.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:01 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:02.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:01 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:02.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:01 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:13:02.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:01 smithi178 ceph-mon[33233]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:02.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:01 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:02.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:01 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:02.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:01 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:13:02.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:01 smithi104 ceph-mon[32726]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:02.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:01 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:02.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:01 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:02.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:01 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:13:04.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:03 smithi178 ceph-mon[33233]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:04.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:03 smithi038 ceph-mon[29400]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:04.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:03 smithi104 ceph-mon[32726]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:06.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:05 smithi178 ceph-mon[33233]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:13:06.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:05 smithi038 ceph-mon[29400]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:13:06.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:05 smithi104 ceph-mon[32726]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:13:08.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:07 smithi178 ceph-mon[33233]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:13:08.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:07 smithi038 ceph-mon[29400]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:13:08.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:07 smithi104 ceph-mon[32726]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:13:10.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:09 smithi178 ceph-mon[33233]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:13:10.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:09 smithi038 ceph-mon[29400]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:13:10.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:09 smithi104 ceph-mon[32726]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:13:12.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:11 smithi178 ceph-mon[33233]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:13:12.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:11 smithi178 ceph-mon[33233]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:11 smithi038 ceph-mon[29400]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:13:12.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:11 smithi038 ceph-mon[29400]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:13:12.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:11 smithi104 ceph-mon[32726]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:13:12.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:11 smithi104 ceph-mon[32726]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:13:14.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:13 smithi178 ceph-mon[33233]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:13:14.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:13 smithi038 ceph-mon[29400]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:13:14.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:13 smithi104 ceph-mon[32726]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:13:16.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:15 smithi178 ceph-mon[33233]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:13:16.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:15 smithi178 ceph-mon[33233]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:13:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:15 smithi038 ceph-mon[29400]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:13:16.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:15 smithi038 ceph-mon[29400]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:13:16.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:15 smithi104 ceph-mon[32726]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:13:16.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:15 smithi104 ceph-mon[32726]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:13:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:17 smithi178 ceph-mon[33233]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:13:18.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:17 smithi038 ceph-mon[29400]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:13:18.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:17 smithi104 ceph-mon[32726]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:13:20.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:19 smithi178 ceph-mon[33233]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:13:20.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:19 smithi038 ceph-mon[29400]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:13:20.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:19 smithi104 ceph-mon[32726]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:13:22.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:21 smithi038 ceph-mon[29400]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 209 B/s wr, 10 op/s 2024-04-25T13:13:22.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:21 smithi104 ceph-mon[32726]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 209 B/s wr, 10 op/s 2024-04-25T13:13:22.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:21 smithi178 ceph-mon[33233]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 209 B/s wr, 10 op/s 2024-04-25T13:13:24.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:23 smithi178 ceph-mon[33233]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:13:24.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:23 smithi038 ceph-mon[29400]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:13:24.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:23 smithi104 ceph-mon[32726]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:13:26.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:26 smithi038 ceph-mon[29400]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-25T13:13:26.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:26 smithi104 ceph-mon[32726]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-25T13:13:26.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:26 smithi178 ceph-mon[33233]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-25T13:13:28.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:28 smithi038 ceph-mon[29400]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 94 B/s wr, 9 op/s 2024-04-25T13:13:28.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:28 smithi104 ceph-mon[32726]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 94 B/s wr, 9 op/s 2024-04-25T13:13:28.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:28 smithi178 ceph-mon[33233]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 94 B/s wr, 9 op/s 2024-04-25T13:13:30.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:30 smithi038 ceph-mon[29400]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:13:30.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:30 smithi104 ceph-mon[32726]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:13:30.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:30 smithi178 ceph-mon[33233]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:13:32.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:32 smithi038 ceph-mon[29400]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:13:32.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:32 smithi104 ceph-mon[32726]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:13:32.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:32 smithi178 ceph-mon[33233]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:13:34.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:34 smithi038 ceph-mon[29400]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:34.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:34 smithi104 ceph-mon[32726]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:34.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:34 smithi178 ceph-mon[33233]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:36.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:36 smithi038 ceph-mon[29400]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T13:13:36.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:36 smithi104 ceph-mon[32726]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T13:13:36.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:36 smithi178 ceph-mon[33233]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T13:13:38.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:38 smithi038 ceph-mon[29400]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:38.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:38 smithi104 ceph-mon[32726]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:38.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:38 smithi178 ceph-mon[33233]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:40.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:40 smithi038 ceph-mon[29400]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:40.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:40 smithi104 ceph-mon[32726]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:40.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:40 smithi178 ceph-mon[33233]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T13:13:42.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:42 smithi038 ceph-mon[29400]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-04-25T13:13:42.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:42 smithi104 ceph-mon[32726]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-04-25T13:13:42.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:42 smithi178 ceph-mon[33233]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-04-25T13:13:44.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:44 smithi038 ceph-mon[29400]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:44.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:44 smithi104 ceph-mon[32726]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:44.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:44 smithi178 ceph-mon[33233]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:46.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:46 smithi038 ceph-mon[29400]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:46.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:46 smithi104 ceph-mon[32726]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:46.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:46 smithi178 ceph-mon[33233]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:48.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:48 smithi178 ceph-mon[33233]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:48.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:48 smithi038 ceph-mon[29400]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:48.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:48 smithi104 ceph-mon[32726]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:50.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:50 smithi178 ceph-mon[33233]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:50.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:50 smithi038 ceph-mon[29400]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:50.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:50 smithi104 ceph-mon[32726]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:52.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:52 smithi178 ceph-mon[33233]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:52.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:52 smithi038 ceph-mon[29400]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:52.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:52 smithi104 ceph-mon[32726]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:54.377 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:54 smithi104 ceph-mon[32726]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:54.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:54 smithi178 ceph-mon[33233]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:54.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:54 smithi038 ceph-mon[29400]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:56.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:56 smithi178 ceph-mon[33233]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:56.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:56 smithi038 ceph-mon[29400]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:56.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:56 smithi104 ceph-mon[32726]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:13:58.494 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:13:58 smithi178 ceph-mon[33233]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:58.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:13:58 smithi038 ceph-mon[29400]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:13:58.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:13:58 smithi104 ceph-mon[32726]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:00.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:00 smithi178 ceph-mon[33233]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:00.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:00 smithi038 ceph-mon[29400]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:00.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:00 smithi104 ceph-mon[32726]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:02.375 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:02 smithi038 ceph-mon[29400]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:02.375 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:02 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:02.375 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:02 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:02.375 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:02 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:02.375 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:02 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:14:02.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:02 smithi178 ceph-mon[33233]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:02.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:02 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:02.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:02 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:02.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:02 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:02.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:02 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:14:02.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:02 smithi104 ceph-mon[32726]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:02.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:02 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:02.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:02 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:02.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:02 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:02.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:02 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:14:04.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:04 smithi178 ceph-mon[33233]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:04.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:04 smithi038 ceph-mon[29400]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:04.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:04 smithi104 ceph-mon[32726]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:06.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:06 smithi178 ceph-mon[33233]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:14:06.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:06 smithi038 ceph-mon[29400]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:14:06.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:06 smithi104 ceph-mon[32726]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:14:08.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:08 smithi178 ceph-mon[33233]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:14:08.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:08 smithi038 ceph-mon[29400]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:14:08.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:08 smithi104 ceph-mon[32726]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:14:10.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:10 smithi178 ceph-mon[33233]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:14:10.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:10 smithi038 ceph-mon[29400]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:14:10.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:10 smithi104 ceph-mon[32726]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:14:12.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:12 smithi178 ceph-mon[33233]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:14:12.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:12 smithi178 ceph-mon[33233]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:14:12.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:12 smithi038 ceph-mon[29400]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:14:12.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:12 smithi038 ceph-mon[29400]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:14:12.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:12 smithi104 ceph-mon[32726]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:14:12.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:12 smithi104 ceph-mon[32726]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:14:14.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:14 smithi178 ceph-mon[33233]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:14:14.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:14 smithi038 ceph-mon[29400]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:14:14.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:14 smithi104 ceph-mon[32726]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:14:16.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:16 smithi178 ceph-mon[33233]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:14:16.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:16 smithi178 ceph-mon[33233]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:14:16.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:16 smithi038 ceph-mon[29400]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:14:16.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:16 smithi038 ceph-mon[29400]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:14:16.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:16 smithi104 ceph-mon[32726]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:14:16.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:16 smithi104 ceph-mon[32726]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:14:18.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:18 smithi178 ceph-mon[33233]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:14:18.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:18 smithi038 ceph-mon[29400]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:14:18.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:18 smithi104 ceph-mon[32726]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:14:20.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:20 smithi178 ceph-mon[33233]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:14:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:20 smithi038 ceph-mon[29400]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:14:20.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:20 smithi104 ceph-mon[32726]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:14:22.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:22 smithi178 ceph-mon[33233]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-25T13:14:22.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:22 smithi038 ceph-mon[29400]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-25T13:14:22.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:22 smithi104 ceph-mon[32726]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-25T13:14:24.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:24 smithi178 ceph-mon[33233]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:14:24.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:24 smithi038 ceph-mon[29400]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:14:24.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:24 smithi104 ceph-mon[32726]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:14:26.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:26 smithi178 ceph-mon[33233]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:14:26.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:26 smithi038 ceph-mon[29400]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:14:26.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:26 smithi104 ceph-mon[32726]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:14:28.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:28 smithi178 ceph-mon[33233]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 93 B/s wr, 9 op/s 2024-04-25T13:14:28.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:28 smithi038 ceph-mon[29400]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 93 B/s wr, 9 op/s 2024-04-25T13:14:28.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:28 smithi104 ceph-mon[32726]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 93 B/s wr, 9 op/s 2024-04-25T13:14:30.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:30 smithi178 ceph-mon[33233]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:14:30.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:30 smithi038 ceph-mon[29400]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:14:30.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:30 smithi104 ceph-mon[32726]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:14:32.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:32 smithi178 ceph-mon[33233]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:14:32.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:32 smithi038 ceph-mon[29400]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:14:32.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:32 smithi104 ceph-mon[32726]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:14:34.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:34 smithi178 ceph-mon[33233]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:34.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:34 smithi038 ceph-mon[29400]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:34.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:34 smithi104 ceph-mon[32726]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:36.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:36 smithi178 ceph-mon[33233]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:36.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:36 smithi038 ceph-mon[29400]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:36.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:36 smithi104 ceph-mon[32726]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:38.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:38 smithi178 ceph-mon[33233]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:38.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:38 smithi038 ceph-mon[29400]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:38.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:38 smithi104 ceph-mon[32726]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:40.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:40 smithi038 ceph-mon[29400]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:40.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:40 smithi104 ceph-mon[32726]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:40.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:40 smithi178 ceph-mon[33233]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:42.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:42 smithi038 ceph-mon[29400]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:42.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:42 smithi104 ceph-mon[32726]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:42.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:42 smithi178 ceph-mon[33233]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:44.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:44 smithi038 ceph-mon[29400]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:44.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:44 smithi104 ceph-mon[32726]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:44.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:44 smithi178 ceph-mon[33233]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:46.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:46 smithi038 ceph-mon[29400]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:46.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:46 smithi104 ceph-mon[32726]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:46.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:46 smithi178 ceph-mon[33233]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:48.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:48 smithi038 ceph-mon[29400]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:48.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:48 smithi104 ceph-mon[32726]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:48.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:48 smithi178 ceph-mon[33233]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:50.446 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:14:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 -- 172.21.15.38:0/830995371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5690101a90 msgr2=0x7f5690105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 --2- 172.21.15.38:0/830995371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5690101a90 0x7f5690105ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5684009900 tx=0x7f568402a410 comp rx=0 tx=0).stop 2024-04-25T13:14:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 -- 172.21.15.38:0/830995371 shutdown_connections 2024-04-25T13:14:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 --2- 172.21.15.38:0/830995371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56901061b0 0x7f569010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 --2- 172.21.15.38:0/830995371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5690101a90 0x7f5690105ad0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 --2- 172.21.15.38:0/830995371 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 0x7f56901014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 -- 172.21.15.38:0/830995371 >> 172.21.15.38:0/830995371 conn(0x7f56900771d0 msgr2=0x7f56900775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 -- 172.21.15.38:0/830995371 shutdown_connections 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 -- 172.21.15.38:0/830995371 wait complete. 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.563+0000 7f5695b8e640 1 Processor -- start 2024-04-25T13:14:50.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 -- start start 2024-04-25T13:14:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 0x7f569010e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 0x7f569010ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f569010f190 0x7f56901a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5690112350 con 0x7f56901061b0 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56901121d0 con 0x7f56901010e0 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f5695b8e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56901124d0 con 0x7f569010f190 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 0x7f569010ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 0x7f569010ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41862/0 (socket says 172.21.15.38:41862) 2024-04-25T13:14:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 -- 172.21.15.38:0/2628878586 learned_addr learned my addr 172.21.15.38:0/2628878586 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568f7fe640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 0x7f569010e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568ffff640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f569010f190 0x7f56901a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 -- 172.21.15.38:0/2628878586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f569010f190 msgr2=0x7f56901a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f569010f190 0x7f56901a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 -- 172.21.15.38:0/2628878586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 msgr2=0x7f569010e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 0x7f569010e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 -- 172.21.15.38:0/2628878586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56901a7040 con 0x7f56901061b0 2024-04-25T13:14:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568f7fe640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 0x7f569010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.564+0000 7f568effd640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 0x7f569010ec50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5684005940 tx=0x7f56840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.565+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568403f030 con 0x7f56901061b0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.565+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56901a72d0 con 0x7f56901061b0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.565+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5684031910 con 0x7f56901061b0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.565+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684035070 con 0x7f56901061b0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.565+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a77c0 con 0x7f56901061b0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.566+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569004fce0 con 0x7f56901061b0 2024-04-25T13:14:50.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.567+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5684042030 con 0x7f56901061b0 2024-04-25T13:14:50.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.568+0000 7f568cff9640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f56780a3df0 0x7f56780a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.568+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f56840f0c80 con 0x7f56901061b0 2024-04-25T13:14:50.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.568+0000 7f568f7fe640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f56780a3df0 0x7f56780a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.568+0000 7f568f7fe640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f56780a3df0 0x7f56780a62b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56800058c0 tx=0x7f5680005850 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:50.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:50 smithi038 ceph-mon[29400]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:50.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.573+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56840bd310 con 0x7f56901061b0 2024-04-25T13:14:50.574 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:50 smithi104 ceph-mon[32726]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:50 smithi178 ceph-mon[33233]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.783+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 --> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5690109450 con 0x7f56780a3df0 2024-04-25T13:14:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.791+0000 7f568cff9640 1 -- 172.21.15.38:0/2628878586 <== mgr.15345 v2:172.21.15.38:6800/1710923873 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f568000bca0 con 0x7f56780a3df0 2024-04-25T13:14:50.792 INFO:teuthology.orchestra.run.smithi038.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:04:49", "first": "2024-04-25T13:05:00", "last": "2024-04-25T13:14:00", "last_pruned": "2024-04-25T13:14:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T13:14:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f56780a3df0 msgr2=0x7f56780a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f56780a3df0 0x7f56780a62b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56800058c0 tx=0x7f5680005850 comp rx=0 tx=0).stop 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 msgr2=0x7f569010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 0x7f569010ec50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5684005940 tx=0x7f56840042f0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 shutdown_connections 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f56780a3df0 0x7f56780a62b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f569010f190 0x7f56901a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.793+0000 7f5695b8e640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56901061b0 0x7f569010ec50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.794+0000 7f5695b8e640 1 --2- 172.21.15.38:0/2628878586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56901010e0 0x7f569010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.794+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 >> 172.21.15.38:0/2628878586 conn(0x7f56900771d0 msgr2=0x7f56900febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.794+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 shutdown_connections 2024-04-25T13:14:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.794+0000 7f5695b8e640 1 -- 172.21.15.38:0/2628878586 wait complete. 2024-04-25T13:14:50.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:14:50.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4263895652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0102b80 msgr2=0x7fb6a0106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4263895652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0102b80 0x7fb6a0106bc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb694009900 tx=0x7fb69402a410 comp rx=0 tx=0).stop 2024-04-25T13:14:50.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4263895652 shutdown_connections 2024-04-25T13:14:50.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4263895652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6a01072a0 0x7fb6a010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4263895652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0102b80 0x7fb6a0106bc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4263895652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01021d0 0x7fb6a01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4263895652 >> 172.21.15.38:0/4263895652 conn(0x7fb6a00793e0 msgr2=0x7fb6a00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:50.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4263895652 shutdown_connections 2024-04-25T13:14:50.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.908+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4263895652 wait complete. 2024-04-25T13:14:50.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 Processor -- start 2024-04-25T13:14:50.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 -- start start 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6a01021d0 0x7fb6a0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01072a0 0x7fb6a0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 0x7fb6a01a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a0112320 con 0x7fb6a0074cd0 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01121a0 con 0x7fb6a01072a0 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb6a4f22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01124a0 con 0x7fb6a01021d0 2024-04-25T13:14:50.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb69ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 0x7fb6a01a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb69ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 0x7fb6a01a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41876/0 (socket says 172.21.15.38:41876) 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.909+0000 7fb69ed9d640 1 -- 172.21.15.38:0/4011923397 learned_addr learned my addr 172.21.15.38:0/4011923397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69e59c640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6a01021d0 0x7fb6a0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69dd9b640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01072a0 0x7fb6a0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69ed9d640 1 -- 172.21.15.38:0/4011923397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6a01021d0 msgr2=0x7fb6a0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69ed9d640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6a01021d0 0x7fb6a0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69ed9d640 1 -- 172.21.15.38:0/4011923397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01072a0 msgr2=0x7fb6a0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:50.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69ed9d640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01072a0 0x7fb6a0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69ed9d640 1 -- 172.21.15.38:0/4011923397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a01a7050 con 0x7fb6a0074cd0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69dd9b640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01072a0 0x7fb6a0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb69ed9d640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 0x7fb6a01a6950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb69800cab0 tx=0x7fb69800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69800adc0 con 0x7fb6a0074cd0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb69800f920 con 0x7fb6a0074cd0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a01a7340 con 0x7fb6a0074cd0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698018aa0 con 0x7fb6a0074cd0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.910+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a01a7890 con 0x7fb6a0074cd0 2024-04-25T13:14:50.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.911+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a004fce0 con 0x7fb6a0074cd0 2024-04-25T13:14:50.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.912+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb69800fad0 con 0x7fb6a0074cd0 2024-04-25T13:14:50.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.913+0000 7fb68f7fe640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fb6880a3df0 0x7fb6880a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:50.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.913+0000 7fb69e59c640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fb6880a3df0 0x7fb6880a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:50.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.913+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb6980d0ff0 con 0x7fb6a0074cd0 2024-04-25T13:14:50.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.913+0000 7fb69e59c640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fb6880a3df0 0x7fb6880a62b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb690005fd0 tx=0x7fb690005850 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:50.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:50.917+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb698014030 con 0x7fb6a0074cd0 2024-04-25T13:14:51.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.126+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 --> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb6a0102b80 con 0x7fb6880a3df0 2024-04-25T13:14:51.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.133+0000 7fb68f7fe640 1 -- 172.21.15.38:0/4011923397 <== mgr.15345 v2:172.21.15.38:6800/1710923873 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb6a0102b80 con 0x7fb6880a3df0 2024-04-25T13:14:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/ 1m 6m3h 2024-04-25T13:14:51.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fb6880a3df0 msgr2=0x7fb6880a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fb6880a3df0 0x7fb6880a62b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb690005fd0 tx=0x7fb690005850 comp rx=0 tx=0).stop 2024-04-25T13:14:51.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 msgr2=0x7fb6a01a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 0x7fb6a01a6950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb69800cab0 tx=0x7fb69800cf70 comp rx=0 tx=0).stop 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 shutdown_connections 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0074cd0 0x7fb6a01a6950 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6a01072a0 0x7fb6a0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fb6880a3df0 0x7fb6880a62b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 --2- 172.21.15.38:0/4011923397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6a01021d0 0x7fb6a0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 >> 172.21.15.38:0/4011923397 conn(0x7fb6a00793e0 msgr2=0x7fb6a00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 shutdown_connections 2024-04-25T13:14:51.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.135+0000 7fb6a4f22640 1 -- 172.21.15.38:0/4011923397 wait complete. 2024-04-25T13:14:51.146 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T13:14:51.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:14:51.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 -- 172.21.15.38:0/2725958384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 msgr2=0x7f4208103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 --2- 172.21.15.38:0/2725958384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f4208103610 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4204009900 tx=0x7f420402a480 comp rx=0 tx=0).stop 2024-04-25T13:14:51.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 -- 172.21.15.38:0/2725958384 shutdown_connections 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 --2- 172.21.15.38:0/2725958384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f420810c570 0x7f420810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 --2- 172.21.15.38:0/2725958384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4208103b50 0x7f420810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 --2- 172.21.15.38:0/2725958384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f4208103610 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 -- 172.21.15.38:0/2725958384 >> 172.21.15.38:0/2725958384 conn(0x7f4208077fb0 msgr2=0x7f42080783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 -- 172.21.15.38:0/2725958384 shutdown_connections 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.259+0000 7f420fd4b640 1 -- 172.21.15.38:0/2725958384 wait complete. 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 Processor -- start 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 -- start start 2024-04-25T13:14:51.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f420819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4208103b50 0x7f420819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420810c570 0x7f4208198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4208112310 con 0x7f4208103230 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4208112190 con 0x7f420810c570 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420fd4b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4208112490 con 0x7f4208103b50 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420dae7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f420819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420dae7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f420819e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41900/0 (socket says 172.21.15.38:41900) 2024-04-25T13:14:51.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420dae7640 1 -- 172.21.15.38:0/1749878106 learned_addr learned my addr 172.21.15.38:0/1749878106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420e2e8640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420810c570 0x7f4208198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420d2e6640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4208103b50 0x7f420819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420dae7640 1 -- 172.21.15.38:0/1749878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4208103b50 msgr2=0x7f420819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.260+0000 7f420dae7640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4208103b50 0x7f420819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420dae7640 1 -- 172.21.15.38:0/1749878106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420810c570 msgr2=0x7f4208198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420dae7640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420810c570 0x7f4208198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420dae7640 1 -- 172.21.15.38:0/1749878106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4208198f50 con 0x7f4208103230 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420d2e6640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4208103b50 0x7f420819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:14:51.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420dae7640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f420819e580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4204005ab0 tx=0x7f42040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420403f030 con 0x7f4208103230 2024-04-25T13:14:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42081991e0 con 0x7f4208103230 2024-04-25T13:14:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4204003cd0 con 0x7f4208103230 2024-04-25T13:14:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204035070 con 0x7f4208103230 2024-04-25T13:14:51.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.261+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42081a5320 con 0x7f4208103230 2024-04-25T13:14:51.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.262+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420806ad80 con 0x7f4208103230 2024-04-25T13:14:51.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.263+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4204042030 con 0x7f4208103230 2024-04-25T13:14:51.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.264+0000 7f41faffd640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f41f40a3d20 0x7f41f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.264+0000 7f420d2e6640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f41f40a3d20 0x7f41f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.264+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f42040f1340 con 0x7f4208103230 2024-04-25T13:14:51.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.264+0000 7f420d2e6640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f41f40a3d20 0x7f41f40a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4208199e90 tx=0x7f41fc005f90 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:51.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.267+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42040bd970 con 0x7f4208103230 2024-04-25T13:14:51.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.457+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42081076f0 con 0x7f4208103230 2024-04-25T13:14:51.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.459+0000 7f41faffd640 1 -- 172.21.15.38:0/1749878106 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23117 (secure 0 0 0) 0x7f42040bd270 con 0x7f4208103230 2024-04-25T13:14:51.460 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:14:51.460 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":97,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:14:15.847050+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/448455346":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1381358988":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/3705953772":"2024-04-26T13:04:48.566935+0000","172.21.15.38:6801/1333793113":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1475454542":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/1333793113":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:14:51.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f41f40a3d20 msgr2=0x7f41f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f41f40a3d20 0x7f41f40a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4208199e90 tx=0x7f41fc005f90 comp rx=0 tx=0).stop 2024-04-25T13:14:51.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 msgr2=0x7f420819e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f420819e580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4204005ab0 tx=0x7f42040042f0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 shutdown_connections 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420810c570 0x7f4208198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f41f40a3d20 0x7f41f40a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4208103b50 0x7f420819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 --2- 172.21.15.38:0/1749878106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4208103230 0x7f420819e580 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 >> 172.21.15.38:0/1749878106 conn(0x7f4208077fb0 msgr2=0x7f42080fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 shutdown_connections 2024-04-25T13:14:51.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.461+0000 7f420fd4b640 1 -- 172.21.15.38:0/1749878106 wait complete. 2024-04-25T13:14:51.473 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:02:42.235731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:02:43.206179+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:14:51.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:14:51.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 -- 172.21.15.38:0/2990368392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 msgr2=0x7f5e701035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 --2- 172.21.15.38:0/2990368392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e701035b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c02a510 comp rx=0 tx=0).stop 2024-04-25T13:14:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 -- 172.21.15.38:0/2990368392 shutdown_connections 2024-04-25T13:14:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 --2- 172.21.15.38:0/2990368392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e7010c510 0x7f5e7010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 --2- 172.21.15.38:0/2990368392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70103af0 0x7f5e7010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 --2- 172.21.15.38:0/2990368392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e701035b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 -- 172.21.15.38:0/2990368392 >> 172.21.15.38:0/2990368392 conn(0x7f5e70077f50 msgr2=0x7f5e70078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 -- 172.21.15.38:0/2990368392 shutdown_connections 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 -- 172.21.15.38:0/2990368392 wait complete. 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 Processor -- start 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.578+0000 7f5e77f07640 1 -- start start 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e7019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e70103af0 0x7f5e7019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7010c510 0x7f5e70198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701122e0 con 0x7f5e701031d0 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70112160 con 0x7f5e70103af0 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70112460 con 0x7f5e7010c510 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e7019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e7019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41920/0 (socket says 172.21.15.38:41920) 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 -- 172.21.15.38:0/329405835 learned_addr learned my addr 172.21.15.38:0/329405835 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e754a2640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e70103af0 0x7f5e7019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e764a4640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7010c510 0x7f5e70198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 -- 172.21.15.38:0/329405835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7010c510 msgr2=0x7f5e70198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7010c510 0x7f5e70198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 -- 172.21.15.38:0/329405835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e70103af0 msgr2=0x7f5e7019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e70103af0 0x7f5e7019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 -- 172.21.15.38:0/329405835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e70198f40 con 0x7f5e701031d0 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e75ca3640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e7019e5c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c03f070 con 0x7f5e701031d0 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e701991d0 con 0x7f5e701031d0 2024-04-25T13:14:51.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5e6c031e10 con 0x7f5e701031d0 2024-04-25T13:14:51.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.579+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c035030 con 0x7f5e701031d0 2024-04-25T13:14:51.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.580+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e701a5540 con 0x7f5e701031d0 2024-04-25T13:14:51.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.581+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e700698e0 con 0x7f5e701031d0 2024-04-25T13:14:51.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.582+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e6c042070 con 0x7f5e701031d0 2024-04-25T13:14:51.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.583+0000 7f5e66ffd640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f5e5c0a3d20 0x7f5e5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.583+0000 7f5e754a2640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f5e5c0a3d20 0x7f5e5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.583+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5e6c0f1870 con 0x7f5e701031d0 2024-04-25T13:14:51.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.583+0000 7f5e754a2640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f5e5c0a3d20 0x7f5e5c0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e70199ed0 tx=0x7f5e6000a250 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:51.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.587+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e6c0bdf00 con 0x7f5e701031d0 2024-04-25T13:14:51.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.773+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e70199530 con 0x7f5e701031d0 2024-04-25T13:14:51.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.774+0000 7f5e66ffd640 1 -- 172.21.15.38:0/329405835 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f5e6c0bd800 con 0x7f5e701031d0 2024-04-25T13:14:51.775 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f5e5c0a3d20 msgr2=0x7f5e5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f5e5c0a3d20 0x7f5e5c0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e70199ed0 tx=0x7f5e6000a250 comp rx=0 tx=0).stop 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 msgr2=0x7f5e7019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e7019e5c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c004430 comp rx=0 tx=0).stop 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 shutdown_connections 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f5e5c0a3d20 0x7f5e5c0a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7010c510 0x7f5e70198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e70103af0 0x7f5e7019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 --2- 172.21.15.38:0/329405835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e701031d0 0x7f5e7019e5c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 >> 172.21.15.38:0/329405835 conn(0x7f5e70077f50 msgr2=0x7f5e700ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 shutdown_connections 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.776+0000 7f5e77f07640 1 -- 172.21.15.38:0/329405835 wait complete. 2024-04-25T13:14:51.788 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:14:51.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 -- 172.21.15.38:0/75790576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b41003e0 msgr2=0x7f20b4100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/75790576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b41003e0 0x7f20b4100820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f209c009930 tx=0x7f209c02a420 comp rx=0 tx=0).stop 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 -- 172.21.15.38:0/75790576 shutdown_connections 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/75790576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 0x7f20b410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/75790576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b41003e0 0x7f20b4100820 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/75790576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b4107fd0 0x7f20b41083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 -- 172.21.15.38:0/75790576 >> 172.21.15.38:0/75790576 conn(0x7f20b4077360 msgr2=0x7f20b40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:51.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 -- 172.21.15.38:0/75790576 shutdown_connections 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.892+0000 7f20b9a3c640 1 -- 172.21.15.38:0/75790576 wait complete. 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 Processor -- start 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 -- start start 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b41003e0 0x7f20b41015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 0x7f20b4101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 0x7f20b4102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4112360 con 0x7f20b4107fd0 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41121e0 con 0x7f20b4100d60 2024-04-25T13:14:51.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b9a3c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41124e0 con 0x7f20b41003e0 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 0x7f20b4102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 0x7f20b4102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41936/0 (socket says 172.21.15.38:41936) 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20b37fe640 1 -- 172.21.15.38:0/2237192483 learned_addr learned my addr 172.21.15.38:0/2237192483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.893+0000 7f20abfff640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 0x7f20b4101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b2ffd640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b41003e0 0x7f20b41015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b37fe640 1 -- 172.21.15.38:0/2237192483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b41003e0 msgr2=0x7f20b41015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b37fe640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b41003e0 0x7f20b41015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b37fe640 1 -- 172.21.15.38:0/2237192483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 msgr2=0x7f20b4101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b37fe640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 0x7f20b4101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b37fe640 1 -- 172.21.15.38:0/2237192483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41a6e70 con 0x7f20b4107fd0 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20abfff640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 0x7f20b4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b37fe640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 0x7f20b4102040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00eb10 tx=0x7f20ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac004b20 con 0x7f20b4107fd0 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41a7160 con 0x7f20b4107fd0 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f20ac004cc0 con 0x7f20b4107fd0 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac010720 con 0x7f20b4107fd0 2024-04-25T13:14:51.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.894+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a7630 con 0x7f20b4107fd0 2024-04-25T13:14:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.895+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b4102af0 con 0x7f20b4107fd0 2024-04-25T13:14:51.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.896+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20ac0108c0 con 0x7f20b4107fd0 2024-04-25T13:14:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.897+0000 7f20b8a3a640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f20a00a3d20 0x7f20a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.897+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f20ac0d1110 con 0x7f20b4107fd0 2024-04-25T13:14:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.897+0000 7f20b2ffd640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f20a00a3d20 0x7f20a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.898+0000 7f20b2ffd640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f20a00a3d20 0x7f20a00a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a4005850 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:51.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:51.901+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20ac053ff0 con 0x7f20b4107fd0 2024-04-25T13:14:52.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.088+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20b410a450 con 0x7f20b4107fd0 2024-04-25T13:14:52.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.088+0000 7f20b8a3a640 1 -- 172.21.15.38:0/2237192483 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f20ac09d7a0 con 0x7f20b4107fd0 2024-04-25T13:14:52.089 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T13:14:52.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f20a00a3d20 msgr2=0x7f20a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f20a00a3d20 0x7f20a00a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a4005850 comp rx=0 tx=0).stop 2024-04-25T13:14:52.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 msgr2=0x7f20b4102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 0x7f20b4102040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00eb10 tx=0x7f20ac00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 shutdown_connections 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4107fd0 0x7f20b4102040 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20b4100d60 0x7f20b4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f20a00a3d20 0x7f20a00a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 --2- 172.21.15.38:0/2237192483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b41003e0 0x7f20b41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 >> 172.21.15.38:0/2237192483 conn(0x7f20b4077360 msgr2=0x7f20b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 shutdown_connections 2024-04-25T13:14:52.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.090+0000 7f20b9a3c640 1 -- 172.21.15.38:0/2237192483 wait complete. 2024-04-25T13:14:52.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 -- 172.21.15.38:0/316249079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434107260 msgr2=0x7f843410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 --2- 172.21.15.38:0/316249079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434107260 0x7f843410ee30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f842c009ca0 tx=0x7f842c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 -- 172.21.15.38:0/316249079 shutdown_connections 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 --2- 172.21.15.38:0/316249079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434107260 0x7f843410ee30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 --2- 172.21.15.38:0/316249079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8434102b40 0x7f8434106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 --2- 172.21.15.38:0/316249079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8434102190 0x7f8434102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 -- 172.21.15.38:0/316249079 >> 172.21.15.38:0/316249079 conn(0x7f84340fd990 msgr2=0x7f84340ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 -- 172.21.15.38:0/316249079 shutdown_connections 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.206+0000 7f843c56f640 1 -- 172.21.15.38:0/316249079 wait complete. 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 Processor -- start 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 -- start start 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 0x7f843419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8434107260 0x7f843419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8434198610 0x7f8434198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112330 con 0x7f8434102190 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f84341121b0 con 0x7f8434198610 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843c56f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84341124b0 con 0x7f8434107260 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 0x7f843419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 0x7f843419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41958/0 (socket says 172.21.15.38:41958) 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 -- 172.21.15.38:0/974207904 learned_addr learned my addr 172.21.15.38:0/974207904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f8439b0a640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8434107260 0x7f843419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843ab0c640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8434198610 0x7f8434198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 -- 172.21.15.38:0/974207904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8434107260 msgr2=0x7f843419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8434107260 0x7f843419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 -- 172.21.15.38:0/974207904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8434198610 msgr2=0x7f8434198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8434198610 0x7f8434198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 -- 172.21.15.38:0/974207904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8434199130 con 0x7f8434102190 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.207+0000 7f843a30b640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 0x7f843419e460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f843000e9f0 tx=0x7f843000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.208+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000cd80 con 0x7f8434102190 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.208+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8430010900 con 0x7f8434102190 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.208+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8434199420 con 0x7f8434102190 2024-04-25T13:14:52.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.208+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430018920 con 0x7f8434102190 2024-04-25T13:14:52.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.208+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a56c0 con 0x7f8434102190 2024-04-25T13:14:52.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.209+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84340691c0 con 0x7f8434102190 2024-04-25T13:14:52.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.210+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8430010ab0 con 0x7f8434102190 2024-04-25T13:14:52.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.210+0000 7f842b7fe640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f84200a3da0 0x7f84200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.211+0000 7f8439b0a640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f84200a3da0 0x7f84200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.211+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f84300d0b20 con 0x7f8434102190 2024-04-25T13:14:52.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.211+0000 7f8439b0a640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f84200a3da0 0x7f84200a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8434108830 tx=0x7f842400b390 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.215+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f843009cb60 con 0x7f8434102190 2024-04-25T13:14:52.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.402+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f843410a520 con 0x7f8434102190 2024-04-25T13:14:52.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.402+0000 7f842b7fe640 1 -- 172.21.15.38:0/974207904 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f84300bd0f0 con 0x7f8434102190 2024-04-25T13:14:52.403 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T13:14:52.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f84200a3da0 msgr2=0x7f84200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f84200a3da0 0x7f84200a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8434108830 tx=0x7f842400b390 comp rx=0 tx=0).stop 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 msgr2=0x7f843419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 0x7f843419e460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f843000e9f0 tx=0x7f843000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 shutdown_connections 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8434198610 0x7f8434198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f84200a3da0 0x7f84200a6260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8434107260 0x7f843419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 --2- 172.21.15.38:0/974207904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8434102190 0x7f843419e460 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.404+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 >> 172.21.15.38:0/974207904 conn(0x7f84340fd990 msgr2=0x7f84340fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.405+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 shutdown_connections 2024-04-25T13:14:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.405+0000 7f843c56f640 1 -- 172.21.15.38:0/974207904 wait complete. 2024-04-25T13:14:52.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:52 smithi038 ceph-mon[29400]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:52.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:52 smithi038 ceph-mon[29400]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:52.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:52 smithi038 ceph-mon[29400]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:52.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:52 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1749878106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:14:52.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:52 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/329405835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:14:52.415 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:52 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2237192483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:14:52.417 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T13:14:52.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 -- 172.21.15.38:0/3770033215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f24102240 msgr2=0x7f2f24102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/3770033215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f24102240 0x7f2f24102620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009900 tx=0x7f2f0c02a410 comp rx=0 tx=0).stop 2024-04-25T13:14:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 -- 172.21.15.38:0/3770033215 shutdown_connections 2024-04-25T13:14:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/3770033215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f24107280 0x7f2f2410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/3770033215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102bf0 0x7f2f24106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/3770033215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f24102240 0x7f2f24102620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 -- 172.21.15.38:0/3770033215 >> 172.21.15.38:0/3770033215 conn(0x7f2f240790e0 msgr2=0x7f2f240794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 -- 172.21.15.38:0/3770033215 shutdown_connections 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.517+0000 7f2f2a88a640 1 -- 172.21.15.38:0/3770033215 wait complete. 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 Processor -- start 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 -- start start 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102240 0x7f2f2419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f24107280 0x7f2f2419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 0x7f2f24198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24112340 con 0x7f2f241986a0 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241121c0 con 0x7f2f24107280 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f2a88a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241124c0 con 0x7f2f24102240 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f28e27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 0x7f2f24198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f28e27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 0x7f2f24198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41982/0 (socket says 172.21.15.38:41982) 2024-04-25T13:14:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f28e27640 1 -- 172.21.15.38:0/35147478 learned_addr learned my addr 172.21.15.38:0/35147478 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f237fe640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f24107280 0x7f2f2419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.518+0000 7f2f23fff640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102240 0x7f2f2419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f28e27640 1 -- 172.21.15.38:0/35147478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102240 msgr2=0x7f2f2419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f28e27640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102240 0x7f2f2419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f28e27640 1 -- 172.21.15.38:0/35147478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f24107280 msgr2=0x7f2f2419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f28e27640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f24107280 0x7f2f2419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f28e27640 1 -- 172.21.15.38:0/35147478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f241992a0 con 0x7f2f241986a0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f28e27640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 0x7f2f24198b20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00cad0 tx=0x7f2f1c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00ae20 con 0x7f2f241986a0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2f1c005020 con 0x7f2f241986a0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f241a5380 con 0x7f2f241986a0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00f6b0 con 0x7f2f241986a0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.519+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f241a5780 con 0x7f2f241986a0 2024-04-25T13:14:52.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.520+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f24103680 con 0x7f2f241986a0 2024-04-25T13:14:52.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.521+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f1c0051d0 con 0x7f2f241986a0 2024-04-25T13:14:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.522+0000 7f2f217fa640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f2f100a3da0 0x7f2f100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.522+0000 7f2f23fff640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f2f100a3da0 0x7f2f100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.522+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f2f1c0d0710 con 0x7f2f241986a0 2024-04-25T13:14:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.522+0000 7f2f23fff640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f2f100a3da0 0x7f2f100a6260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009230 tx=0x7f2f0c0058d0 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.526+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f1c014030 con 0x7f2f241986a0 2024-04-25T13:14:52.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:52 smithi104 ceph-mon[32726]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:52.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:52 smithi104 ceph-mon[32726]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:52.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:52 smithi104 ceph-mon[32726]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:52.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:52 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1749878106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:14:52.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:52 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/329405835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:14:52.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:52 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2237192483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:14:52.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:52 smithi178 ceph-mon[33233]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T13:14:52.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:52 smithi178 ceph-mon[33233]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:52.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:52 smithi178 ceph-mon[33233]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:52.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:52 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1749878106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:14:52.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:52 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/329405835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:14:52.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:52 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2237192483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:14:52.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.767+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2f2410a480 con 0x7f2f241986a0 2024-04-25T13:14:52.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.776+0000 7f2f217fa640 1 -- 172.21.15.38:0/35147478 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f2f1c09c750 con 0x7f2f241986a0 2024-04-25T13:14:52.788 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:14:52.788 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:14:52.788 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 26, 2024-04-25T13:14:52.788 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.788 INFO:teuthology.orchestra.run.smithi038.stdout: "active_gid": 15345, 2024-04-25T13:14:52.788 INFO:teuthology.orchestra.run.smithi038.stdout: "active_name": "x", 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addrs": { 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6800", 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1710923873 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.789 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6801", 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1710923873 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addr": "172.21.15.38:6801/1710923873", 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: "active_change": "2024-04-25T13:04:48.567267+0000", 2024-04-25T13:14:52.790 INFO:teuthology.orchestra.run.smithi038.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "available": true, 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "standbys": [ 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24949, 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "z", 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.792 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.793 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.794 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.797 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.798 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.800 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.802 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.803 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.804 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.805 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.806 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.807 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-04-25T13:14:52.808 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-04-25T13:14:52.809 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.810 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.811 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.812 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-04-25T13:14:52.813 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-04-25T13:14:52.814 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:52.815 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-04-25T13:14:52.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.818 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.819 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.820 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:52.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-04-25T13:14:52.822 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:14:52.823 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.824 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-04-25T13:14:52.827 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:14:52.828 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.829 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.830 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:52.831 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.833 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.834 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.835 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:52.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.839 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.840 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.843 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-04-25T13:14:52.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:14:52.845 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.846 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.847 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-04-25T13:14:52.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.850 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.851 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-04-25T13:14:52.852 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.853 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.854 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.856 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-04-25T13:14:52.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.859 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.860 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.863 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.864 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.865 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.866 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.867 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.868 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.869 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-04-25T13:14:52.870 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:52.871 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.873 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.874 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.875 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.876 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.878 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.880 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.881 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.882 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.884 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-04-25T13:14:52.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:52.886 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.887 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.888 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.889 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.890 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.893 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.894 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-04-25T13:14:52.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.898 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:52.902 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-04-25T13:14:52.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:14:52.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.907 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.908 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.910 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.911 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:14:52.913 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.914 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.915 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.916 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.917 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.919 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.921 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:52.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.924 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.925 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.926 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.928 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:14:52.931 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:14:52.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.934 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.935 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-04-25T13:14:52.936 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:14:52.937 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.939 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.940 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.941 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.942 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.943 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-04-25T13:14:52.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.945 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.946 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.948 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.949 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.950 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.951 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:52.953 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.954 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.956 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:52.957 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:52.958 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.959 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.960 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.961 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.962 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-04-25T13:14:52.963 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:14:52.964 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:52.966 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:14:52.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.968 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.969 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.970 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:14:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.973 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.974 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.975 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:14:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.978 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.979 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.980 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.981 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:14:52.982 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.983 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.984 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.985 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.986 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.987 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.988 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.989 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.990 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.991 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:14:52.992 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.993 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.994 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.996 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.997 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:52.998 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:52.999 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:14:53.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.001 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.002 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "[]", 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "{}", 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.004 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:14:53.005 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.006 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.007 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.008 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:14:53.010 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.012 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.013 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:14:53.015 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.016 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:14:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:14:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.020 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.021 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.022 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.023 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.024 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:14:53.025 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.026 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.027 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.028 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.029 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:14:53.033 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.034 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.035 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:14:53.038 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.039 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.040 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-04-25T13:14:53.043 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.044 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.045 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.046 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-04-25T13:14:53.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.051 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.053 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.055 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.057 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.058 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.059 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.060 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.061 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.062 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:53.063 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.064 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.065 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.068 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-04-25T13:14:53.069 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-04-25T13:14:53.070 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.071 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.072 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.073 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.075 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.077 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.078 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.081 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.082 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.084 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-04-25T13:14:53.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:14:53.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.089 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.090 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.091 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.092 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.093 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.095 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.096 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.097 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.098 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.099 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.100 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.101 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.102 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.103 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.104 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.106 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.107 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.109 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-04-25T13:14:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:14:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.113 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:53.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:14:53.116 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.118 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.119 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.122 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.123 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.124 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.126 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.127 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-04-25T13:14:53.128 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:14:53.129 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.133 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.135 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.136 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.137 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.138 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.139 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.141 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.142 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-04-25T13:14:53.144 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.146 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.148 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.149 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.150 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.153 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.154 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.155 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-04-25T13:14:53.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.158 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.161 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.162 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.164 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.165 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-04-25T13:14:53.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.168 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.169 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.170 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.171 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.173 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.174 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.175 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.176 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:14:53.177 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.179 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-04-25T13:14:53.180 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-04-25T13:14:53.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.185 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.186 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.189 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.190 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.193 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.194 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.195 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.196 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.197 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.198 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.200 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.201 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.202 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.203 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.204 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.205 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.206 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.207 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.208 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.209 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.210 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.211 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-04-25T13:14:53.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.213 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.214 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.215 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.216 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.217 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.218 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.219 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.220 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.221 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-04-25T13:14:53.222 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.224 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.226 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.227 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.228 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.229 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.230 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.231 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.232 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.234 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.235 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.237 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.238 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.239 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.242 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.243 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.244 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.245 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.248 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.249 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.250 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.251 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.252 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:14:53.255 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.257 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.259 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.260 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.262 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.263 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.264 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.265 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.266 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.267 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.269 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.270 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.274 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.277 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.278 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.279 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.281 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-04-25T13:14:53.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.283 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.284 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.285 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:53.287 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:14:53.288 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.289 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.290 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.291 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.292 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.293 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.295 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.296 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.297 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.299 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.300 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.301 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.302 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.303 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.304 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.305 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.306 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.307 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.308 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.309 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.311 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.312 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.313 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.316 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.318 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.319 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.320 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.321 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:53.322 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.323 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.324 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.325 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.326 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.327 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.328 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.329 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.330 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.331 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.332 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.333 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.334 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.336 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.337 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.338 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.339 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.341 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.342 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.343 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.344 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.345 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.346 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.347 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.349 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.350 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.351 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.352 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.353 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-04-25T13:14:53.354 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.355 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.356 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.357 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.358 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-04-25T13:14:53.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.360 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-04-25T13:14:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.364 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.366 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.367 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.368 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-04-25T13:14:53.370 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.371 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.372 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.373 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.374 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.375 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.377 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.379 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.380 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.381 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.382 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.383 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.384 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.385 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-04-25T13:14:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.387 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.388 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.389 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.390 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.391 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.392 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.393 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.394 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-04-25T13:14:53.395 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.396 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.397 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.398 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.399 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.400 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.401 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.403 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-04-25T13:14:53.404 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.405 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.406 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.407 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.408 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.410 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.411 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.412 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.413 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.414 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.415 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.416 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.417 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.418 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.419 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-04-25T13:14:53.420 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.421 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.422 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.424 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-04-25T13:14:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.426 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.427 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.428 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.429 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.430 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.431 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-04-25T13:14:53.432 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-04-25T13:14:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.434 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.435 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.436 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.437 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.438 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.439 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.440 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.441 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.442 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.443 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.444 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.445 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.446 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.447 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-04-25T13:14:53.448 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.449 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.450 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.451 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.452 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.453 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.455 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.456 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.458 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.459 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.460 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.461 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.462 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.463 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.464 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.465 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.466 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.467 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.468 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.469 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.470 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.471 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.472 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.473 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.474 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.475 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-04-25T13:14:53.477 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.480 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.481 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.482 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.483 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.486 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.487 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.488 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.489 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-04-25T13:14:53.490 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.493 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.494 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:53.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 25034, 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "y", 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-04-25T13:14:53.496 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:53.497 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.498 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.499 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.500 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.501 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.502 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.505 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.507 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.508 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.509 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-04-25T13:14:53.510 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.511 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.512 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.513 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.514 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-04-25T13:14:53.515 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.516 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.517 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.519 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.520 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.521 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.522 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-04-25T13:14:53.523 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.524 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.525 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:53.526 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.527 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:14:53.528 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-04-25T13:14:53.529 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:14:53.530 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.531 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.532 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-04-25T13:14:53.533 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.534 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.535 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.536 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.537 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.538 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.539 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.540 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.541 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.543 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-04-25T13:14:53.544 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-04-25T13:14:53.545 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.547 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.548 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.549 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:53.550 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-04-25T13:14:53.551 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:14:53.552 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-04-25T13:14:53.553 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-04-25T13:14:53.554 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.555 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.556 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.557 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.558 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.559 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-04-25T13:14:53.560 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:14:53.561 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.562 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.563 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.564 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.565 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.566 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.567 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.568 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-04-25T13:14:53.569 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.570 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.571 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.572 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:14:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-04-25T13:14:53.574 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.575 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.576 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.577 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.578 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-04-25T13:14:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:14:53.580 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.581 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.582 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.583 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-04-25T13:14:53.584 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-04-25T13:14:53.585 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.586 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.587 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.588 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.589 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.590 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.591 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-04-25T13:14:53.592 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-04-25T13:14:53.593 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.594 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.595 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.596 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.597 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.598 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.599 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:14:53.600 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.601 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.602 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.603 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.604 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-04-25T13:14:53.605 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:14:53.606 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.607 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.608 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.609 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-04-25T13:14:53.610 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:14:53.611 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.612 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.613 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.614 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.616 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.618 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.619 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.620 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.621 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.622 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.623 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.624 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.625 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.626 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.627 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.628 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.629 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.630 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-04-25T13:14:53.631 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.632 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.633 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.634 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.635 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.636 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.637 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-04-25T13:14:53.638 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.639 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-04-25T13:14:53.640 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:14:53.641 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.642 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.644 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:14:53.645 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:14:53.646 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.647 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.648 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.649 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.650 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.651 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.652 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-04-25T13:14:53.653 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.654 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.655 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.656 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.657 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.658 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.659 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.660 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.664 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.665 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.666 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.667 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-04-25T13:14:53.668 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-04-25T13:14:53.669 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:14:53.670 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-04-25T13:14:53.671 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.672 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.673 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.674 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.675 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.676 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.677 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:14:53.678 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.679 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.680 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.681 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:14:53.683 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.684 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.685 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.686 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.687 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.688 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.689 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.690 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:14:53.691 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.692 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.693 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.694 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.695 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.696 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.697 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.698 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:14:53.699 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.700 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.701 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.703 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.705 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.706 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.708 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.709 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "[]", 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.710 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.711 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "{}", 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.712 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.713 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.714 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.715 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.716 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.717 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.718 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.719 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.721 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.722 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.723 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.725 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.726 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.727 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:14:53.728 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.729 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.730 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.731 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-04-25T13:14:53.733 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.734 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.735 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.736 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.737 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.738 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:14:53.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.741 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.742 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.743 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:14:53.744 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.745 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.746 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-04-25T13:14:53.747 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.748 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.749 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.751 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-04-25T13:14:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.753 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.754 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.755 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.756 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.758 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.759 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.762 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.763 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.765 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.766 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.767 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.768 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.769 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.770 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.771 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-04-25T13:14:53.772 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.773 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.774 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.775 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.777 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.778 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.779 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.780 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.781 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.782 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.783 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.784 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.785 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.786 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.787 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.788 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.789 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.790 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-04-25T13:14:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-04-25T13:14:53.792 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.793 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.794 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.795 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-04-25T13:14:53.797 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.799 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.800 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.802 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.803 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.804 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.805 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.806 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.808 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.810 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.811 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-04-25T13:14:53.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.813 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-04-25T13:14:53.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-04-25T13:14:53.815 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.816 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-04-25T13:14:53.817 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.818 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.819 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.822 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.823 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.824 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.825 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-04-25T13:14:53.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-04-25T13:14:53.829 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:14:53.830 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.831 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.832 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.833 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.834 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.835 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.837 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.839 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.840 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.841 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.842 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.843 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.844 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.846 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.848 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.849 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.850 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.852 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.853 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-04-25T13:14:53.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:14:53.855 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.856 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.857 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.858 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.859 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.862 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.863 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.866 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.867 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.868 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.869 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-04-25T13:14:53.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:14:53.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.882 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.885 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.886 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.887 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.888 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.889 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.890 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.891 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.894 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.895 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.896 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.898 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.903 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.904 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.906 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.907 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.909 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.909 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.909 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.909 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.913 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.914 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.915 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.916 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.917 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.918 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.919 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.921 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-04-25T13:14:53.922 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-04-25T13:14:53.923 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.924 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.926 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.926 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.926 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.926 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.926 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.926 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.928 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.930 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.932 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.933 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.934 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.935 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.936 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.937 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.938 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.939 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.940 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.942 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-04-25T13:14:53.943 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.944 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.945 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.946 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.947 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.948 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.949 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.950 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.951 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.952 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.954 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.954 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.954 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.954 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-04-25T13:14:53.954 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-04-25T13:14:53.954 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:14:53.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.956 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.957 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.958 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.958 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.958 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:14:53.958 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.958 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.958 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.960 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.960 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.960 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.960 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.960 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.960 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.961 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.962 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.963 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.963 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.963 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.963 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.965 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.965 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.965 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.965 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.965 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.965 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.966 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.966 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.966 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.966 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.966 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.966 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.967 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.967 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.967 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.967 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.968 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.969 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-04-25T13:14:53.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.972 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:53.974 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.975 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.976 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.979 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.980 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.981 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.983 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:53.984 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:53.985 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.987 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:53.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.988 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.988 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.988 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.988 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.989 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-04-25T13:14:53.990 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.990 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.990 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.990 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.990 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.990 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.991 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-04-25T13:14:53.992 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.992 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.992 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.992 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.992 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.992 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.993 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-04-25T13:14:53.994 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.994 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.994 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.994 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:14:53.994 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.994 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:53.995 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-04-25T13:14:53.996 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.996 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.996 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.996 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:14:53.996 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.996 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:53.998 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-04-25T13:14:53.998 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:53.998 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:53.998 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:53.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:53.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-04-25T13:14:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-04-25T13:14:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.000 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.000 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.000 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-04-25T13:14:54.000 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.000 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.000 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.001 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-04-25T13:14:54.002 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.003 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.004 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.005 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.006 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.007 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.008 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.009 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.010 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.011 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.012 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.013 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.015 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-04-25T13:14:54.016 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.017 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.018 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.019 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.020 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-04-25T13:14:54.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-04-25T13:14:54.021 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.021 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.021 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.021 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.021 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.022 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.023 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.023 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.023 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.023 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.023 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.023 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.024 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.025 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.028 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.030 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:54.033 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.034 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.036 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.037 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.038 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.039 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.040 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.041 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.044 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.045 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.046 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.047 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.048 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.050 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.051 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.052 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-04-25T13:14:54.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:14:54.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.055 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.056 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.057 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.060 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.061 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.062 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.065 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-04-25T13:14:54.066 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.067 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.069 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-04-25T13:14:54.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.075 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-04-25T13:14:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.094 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.095 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.097 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.098 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.099 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.100 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.101 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.103 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.104 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.105 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.107 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.108 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.109 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.110 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.112 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.113 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.117 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.118 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.121 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.124 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.125 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.126 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.127 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-04-25T13:14:54.129 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.133 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-04-25T13:14:54.134 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:14:54.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.138 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-04-25T13:14:54.139 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.140 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.141 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.142 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.145 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.148 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-04-25T13:14:54.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.150 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.152 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.155 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.156 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-04-25T13:14:54.159 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.160 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.161 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.162 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.164 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.165 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.166 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.167 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.168 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.169 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.170 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.172 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.173 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.175 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.176 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.178 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.180 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.181 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.182 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-04-25T13:14:54.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-04-25T13:14:54.184 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.187 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-04-25T13:14:54.190 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.193 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.196 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.199 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.200 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.202 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.203 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.204 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-04-25T13:14:54.205 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.206 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.207 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.208 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: "modules": [ 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard", 2024-04-25T13:14:54.209 INFO:teuthology.orchestra.run.smithi038.stdout: "iostat", 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: "nfs", 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: "restful", 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: "snap_schedule" 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-04-25T13:14:54.210 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.211 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.212 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.213 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.214 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.219 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.220 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.222 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.223 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-04-25T13:14:54.224 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.225 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.227 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.228 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.229 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:14:54.232 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.233 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.234 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-04-25T13:14:54.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.238 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:54.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-04-25T13:14:54.240 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:14:54.241 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.242 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-04-25T13:14:54.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-04-25T13:14:54.245 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:14:54.246 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.247 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.249 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.250 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.251 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.252 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.253 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.254 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.256 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.257 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.260 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.261 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.262 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.263 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-04-25T13:14:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.265 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.266 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.268 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.269 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.270 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.271 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-04-25T13:14:54.272 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.273 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.274 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.276 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-04-25T13:14:54.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.278 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.279 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.281 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-04-25T13:14:54.282 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:14:54.283 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.284 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.285 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:14:54.286 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-04-25T13:14:54.287 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.288 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.291 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.292 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:14:54.293 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.294 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:14:54.295 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.296 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.297 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:14:54.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.299 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.300 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.301 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-04-25T13:14:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.304 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.305 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.307 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-04-25T13:14:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-04-25T13:14:54.309 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.310 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.311 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:14:54.312 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-04-25T13:14:54.313 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.314 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.315 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.316 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.317 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-04-25T13:14:54.318 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:14:54.319 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.320 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.322 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-04-25T13:14:54.323 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.325 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.326 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-04-25T13:14:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:14:54.331 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.332 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.333 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.334 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.335 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.336 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.337 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.338 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.339 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.340 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.341 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.342 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.343 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.344 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-04-25T13:14:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.346 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.347 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.348 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.349 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-04-25T13:14:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-04-25T13:14:54.351 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.352 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.353 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.354 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.355 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.356 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.357 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.358 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-04-25T13:14:54.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:14:54.360 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.361 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.362 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.363 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-04-25T13:14:54.364 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.365 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.366 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.368 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-04-25T13:14:54.369 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.370 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.371 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.372 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.373 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-04-25T13:14:54.374 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.375 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-04-25T13:14:54.376 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.377 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.378 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.379 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.380 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.381 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.382 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.383 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.384 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-04-25T13:14:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:14:54.386 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.387 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.388 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.389 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.390 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.391 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.392 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.393 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.394 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.395 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.396 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.397 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.398 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.399 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.401 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.402 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.403 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.404 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.405 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:14:54.406 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.407 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.408 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.409 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.410 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:14:54.411 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.412 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.413 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.414 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.415 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.416 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.417 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:54.418 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.420 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.421 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.422 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.423 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:14:54.424 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.425 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.426 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "[]", 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.427 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "{}", 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.428 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:14:54.429 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.430 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.431 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.432 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.433 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:14:54.434 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.435 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.436 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.438 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:14:54.439 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.441 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.442 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.443 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:14:54.444 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.445 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.446 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.447 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.447 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:14:54.447 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.447 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.447 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.447 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.448 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.448 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.448 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.448 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.448 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:14:54.448 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:14:54.449 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.449 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.449 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.449 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.449 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.449 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.450 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.450 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.450 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.450 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.450 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.451 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.451 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:14:54.451 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:14:54.451 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.451 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.451 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.452 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-04-25T13:14:54.452 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.452 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.452 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.452 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.453 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.453 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.453 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.453 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.453 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:14:54.453 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:14:54.454 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.454 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.454 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.454 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-04-25T13:14:54.454 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.455 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.455 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.455 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.455 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.455 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.455 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:14:54.457 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.458 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.459 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.460 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.461 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.462 INFO:teuthology.orchestra.run.smithi038.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.463 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.464 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.465 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.466 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.480 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.481 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.482 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.483 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.484 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.486 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.487 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.488 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.490 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.491 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.493 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-04-25T13:14:54.494 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.495 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.496 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.497 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.498 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.499 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.502 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.503 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.505 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.506 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.507 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.508 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.510 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.511 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-04-25T13:14:54.512 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.513 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.514 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.515 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-04-25T13:14:54.516 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.517 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.518 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-04-25T13:14:54.519 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:14:54.520 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.521 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.522 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.523 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-04-25T13:14:54.525 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.526 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.527 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.528 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.529 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.530 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.531 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.532 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.533 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.534 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.535 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.536 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.537 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-04-25T13:14:54.538 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.539 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.540 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.541 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-04-25T13:14:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:14:54.543 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.544 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.545 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.546 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.547 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.548 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.549 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.550 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.551 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.552 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.553 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.555 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.556 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-04-25T13:14:54.557 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.558 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.559 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.560 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.561 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-04-25T13:14:54.562 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.563 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.564 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.565 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-04-25T13:14:54.566 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.567 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.569 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.570 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.571 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.572 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.573 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.574 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.575 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.577 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.578 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.579 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.580 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.582 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.583 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.584 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.585 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.586 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.587 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-04-25T13:14:54.588 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:14:54.589 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.590 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.591 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.592 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.593 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.594 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.595 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.597 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.598 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.599 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:14:54.600 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.601 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.602 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.603 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.604 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.605 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.606 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.607 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.608 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-04-25T13:14:54.609 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.610 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.611 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.612 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.613 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-04-25T13:14:54.614 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:14:54.615 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.616 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.617 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.618 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.619 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.620 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.621 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.622 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.623 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.624 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.625 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.626 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.627 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.628 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.630 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.631 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.632 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.633 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.634 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.635 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.638 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.639 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.640 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.641 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.642 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.643 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.644 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.645 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.646 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-04-25T13:14:54.647 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.648 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.649 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.650 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.651 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.652 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.653 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.654 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.655 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.656 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-04-25T13:14:54.657 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-04-25T13:14:54.658 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.659 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.660 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.661 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.662 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.663 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.664 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.665 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.666 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.667 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.668 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.669 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.670 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.671 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.672 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.673 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.674 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.675 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.676 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.677 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.678 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.679 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.680 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.681 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.682 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.683 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.684 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.685 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.686 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:54.687 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.688 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.689 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.690 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.691 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.692 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.693 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.694 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.695 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.697 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.698 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.699 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.700 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.702 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.703 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.704 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-04-25T13:14:54.705 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.706 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-04-25T13:14:54.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.708 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.709 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.710 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.711 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.712 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.713 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.714 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.715 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.716 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.717 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.718 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.719 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.720 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.721 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-04-25T13:14:54.722 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.723 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.724 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.725 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-04-25T13:14:54.726 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-04-25T13:14:54.727 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.728 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.729 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.730 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.731 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.733 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.734 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.735 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.736 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.737 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.738 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.739 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.741 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-04-25T13:14:54.742 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.743 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.744 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.745 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.746 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.747 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.748 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-04-25T13:14:54.749 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.750 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.751 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.752 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.753 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.754 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.755 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.756 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.758 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-04-25T13:14:54.759 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.760 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.761 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.762 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.763 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.764 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.765 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.766 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.767 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.768 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.769 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.770 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.771 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-04-25T13:14:54.772 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.773 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.774 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.774 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.774 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.774 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.774 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.774 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.775 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.776 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.777 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.778 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.779 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-04-25T13:14:54.781 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.782 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-04-25T13:14:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.784 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.785 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.786 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.787 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.788 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.789 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.790 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.792 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.793 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.794 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.795 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.797 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.798 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-04-25T13:14:54.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.800 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.801 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.803 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-04-25T13:14:54.804 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.805 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.807 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.808 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-04-25T13:14:54.809 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.810 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.812 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-04-25T13:14:54.813 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.814 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.815 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.817 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.818 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.819 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.821 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.823 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.824 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.826 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.830 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.832 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.833 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.834 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.840 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.841 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.842 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.843 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.844 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.845 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-04-25T13:14:54.846 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.850 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.851 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.854 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.855 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.857 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.858 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-04-25T13:14:54.862 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.864 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.866 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-04-25T13:14:54.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-04-25T13:14:54.872 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.873 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.874 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.878 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.879 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.880 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.881 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.884 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.885 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.886 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.887 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.889 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.891 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-04-25T13:14:54.893 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.894 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.895 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.896 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.902 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.903 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.904 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.906 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.907 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-04-25T13:14:54.910 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.911 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.912 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-04-25T13:14:54.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:14:54.914 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.916 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-04-25T13:14:54.917 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.918 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-04-25T13:14:54.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.921 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.922 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.923 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-04-25T13:14:54.924 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.925 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.926 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-04-25T13:14:54.927 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-04-25T13:14:54.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.931 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-04-25T13:14:54.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-04-25T13:14:54.934 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: "services": { 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard": "https://172.21.15.38:8443/" 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: "always_on_modules": { 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: "octopus": [ 2024-04-25T13:14:54.935 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-04-25T13:14:54.936 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "pacific": [ 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-04-25T13:14:54.937 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "quincy": [ 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-04-25T13:14:54.938 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.939 INFO:teuthology.orchestra.run.smithi038.stdout: "reef": [ 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-04-25T13:14:54.940 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "squid": [ 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-04-25T13:14:54.941 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: "last_failure_osd_epoch": 89, 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: "active_clients": [ 2024-04-25T13:14:54.942 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2505694046 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "libcephsqlite", 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1424783715 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.944 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2517461140 2024-04-25T13:14:54.945 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:54.946 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1635433646 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.947 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 172186682 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-04-25T13:14:54.948 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4056656421 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T13:14:54.949 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:14:54.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:53 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/974207904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:14:54.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:53 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/35147478' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:14:54.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:54 smithi178 ceph-mon[33233]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:54.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:53 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/974207904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:14:54.952 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:53 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/35147478' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:14:54.952 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:54 smithi104 ceph-mon[32726]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:54.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f2f100a3da0 msgr2=0x7f2f100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:54.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f2f100a3da0 0x7f2f100a6260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009230 tx=0x7f2f0c0058d0 comp rx=0 tx=0).stop 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 msgr2=0x7f2f24198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 0x7f2f24198b20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00cad0 tx=0x7f2f1c00cf90 comp rx=0 tx=0).stop 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 shutdown_connections 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f241986a0 0x7f2f24198b20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f24107280 0x7f2f2419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f2f100a3da0 0x7f2f100a6260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 --2- 172.21.15.38:0/35147478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102240 0x7f2f2419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:54.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 >> 172.21.15.38:0/35147478 conn(0x7f2f240790e0 msgr2=0x7f2f241003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:54.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 shutdown_connections 2024-04-25T13:14:54.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:14:52.781+0000 7f2f2a88a640 1 -- 172.21.15.38:0/35147478 wait complete. 2024-04-25T13:14:54.962 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T13:14:54.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T13:14:54.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:53 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/974207904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:14:54.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:53 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/35147478' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:14:54.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:54 smithi038 ceph-mon[29400]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:14:55.223 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:14:56.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:56 smithi038 ceph-mon[29400]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T13:14:56.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:56 smithi104 ceph-mon[32726]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T13:14:56.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:56 smithi178 ceph-mon[33233]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T13:14:57.067 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:14:57.068 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:14:57.068 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 279, 2024-04-25T13:14:57.068 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 472, 2024-04-25T13:14:57.068 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 86646, 2024-04-25T13:14:57.068 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 101685, 2024-04-25T13:14:57.068 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 10, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.054234241, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.021913943, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.023179234, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.005050885, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 470, 2024-04-25T13:14:57.069 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.036536096, 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000077736 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 713, 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000206882, 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000290 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:14:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 68, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 218, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 26680, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 512449, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 16, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 5, 2024-04-25T13:14:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.021334020, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.010974965, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.008744999, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000205456, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 216, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.017134083, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000079324 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:14:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 102, 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000024049, 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000235 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 4240, 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 435760 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 365, 2024-04-25T13:14:57.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 351, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1675823, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 176435, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.054498569, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.016911071, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.046197258, 2024-04-25T13:14:57.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.004688757, 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 350, 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.047106834, 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000134590 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 169, 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000029617, 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000175 2024-04-25T13:14:57.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 1633024, 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 130352 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 48, 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.001250083, 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000026043 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-25T13:14:57.077 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2, 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000006366, 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000003183 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-04-25T13:14:57.078 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-25T13:14:57.079 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 11, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 13, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 13, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-25T13:14:57.080 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 12, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 2, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 2, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 1, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 1, 2024-04-25T13:14:57.081 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 1000, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 1, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-25T13:14:57.082 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 3, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 117, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-25T13:14:57.083 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:14:57.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.085 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 3, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 2, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 3, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:14:57.086 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:14:57.087 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.088 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 4, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 4, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 2, 2024-04-25T13:14:57.089 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 2, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 4195424, 2024-04-25T13:14:57.090 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.006457053, 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002152351 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.091 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 16, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 16, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 15, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 16, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 1, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 13, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 13, 2024-04-25T13:14:57.092 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 54604, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 223608 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-04-25T13:14:57.093 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 4, 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.095 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-25T13:14:57.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.097 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-25T13:14:57.100 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.101 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.101 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.102 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.103 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.104 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-25T13:14:57.105 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.106 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.107 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.108 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.109 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.111 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.112 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 0, 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-04-25T13:14:57.113 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 0, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 88, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:14:57.114 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:14:57.115 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T13:14:57.116 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T13:14:57.117 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 270359, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 89, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:14:57.119 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8128, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 169, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T13:14:57.120 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 64808, 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 235, 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-25T13:14:57.121 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 39, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 11422, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 33, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 39, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 39, 2024-04-25T13:14:57.122 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.215995758, 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005538352 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 39, 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 61 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 39, 2024-04-25T13:14:57.123 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 0, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 39, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 10, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 0, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 3, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 7, 2024-04-25T13:14:57.124 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 6, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 0, 2024-04-25T13:14:57.125 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 20, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-25T13:14:57.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-25T13:14:57.127 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-25T13:14:57.128 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 97, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 11, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 9, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 9, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-25T13:14:57.129 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 14, 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 1 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-25T13:14:57.130 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-25T13:14:57.131 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T13:14:57.132 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 708, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1709122, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 708, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 708, 2024-04-25T13:14:57.133 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1709122, 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-25T13:14:57.134 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 39, 2024-04-25T13:14:57.135 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 11422, 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 39, 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 11422, 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:14:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 39, 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 39, 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 39, 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 39, 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.138 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:14:57.139 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.140 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.141 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:57.141 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x56131d71c500": { 2024-04-25T13:14:57.141 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:14:57.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T13:14:57.141 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 4, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1120, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 4, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3, 2024-04-25T13:14:57.142 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1120, 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:57.143 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=3 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:14:57.975 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:14:57.976 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T13:14:57.976 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T13:14:58.227 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:14:58.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:14:58 smithi038 ceph-mon[29400]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:14:58.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:14:58 smithi104 ceph-mon[32726]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:14:58.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:14:58 smithi178 ceph-mon[33233]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:14:59.886 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 216, 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 205, 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1588064, 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 125984, 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-25T13:14:59.887 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.046790583, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.014533953, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.036041719, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.004727184, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 204, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.010974113, 2024-04-25T13:14:59.888 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000053794 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 1588064, 2024-04-25T13:14:59.889 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 125984 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 157, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 183712, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-04-25T13:14:59.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.014823412, 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.010281675, 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001910457, 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 154, 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.010626115, 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000069000 2024-04-25T13:14:59.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 183712 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.892 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 2992, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 448, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.000672025, 2024-04-25T13:14:59.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000042806, 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000596024, 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000853634, 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000284544 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.895 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:14:59.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:14:59.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:14:59.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:14:59.899 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:14:59.900 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 31, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:14:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-04-25T13:14:59.902 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T13:14:59.903 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 218, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1557623, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 218, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 218, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1557623, 2024-04-25T13:14:59.904 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:14:59.905 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:14:59.905 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:14:59.905 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:14:59.905 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:59.905 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:14:59.905 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:15:00.553 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:00 smithi038 ceph-mon[29400]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:15:00.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:00 smithi104 ceph-mon[32726]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:15:00.583 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T13:15:00.583 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T13:15:00.583 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T13:15:00.584 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T13:15:00.584 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T13:15:00.584 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:15:00.584 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:15:00.584 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T13:15:00.585 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T13:15:00.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:00 smithi178 ceph-mon[33233]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:15:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:15:02.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:02 smithi038 ceph-mon[29400]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T13:15:02.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:02 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:02.540 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:15:02.540 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:02.540 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 735, 2024-04-25T13:15:02.540 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 735, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 248553, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 232907, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.088706325, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.036898190, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.018570194, 2024-04-25T13:15:02.541 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.023034289, 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 734, 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.044744274, 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000060959 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.542 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1349, 2024-04-25T13:15:02.543 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1501, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 344546, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 864467, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 5, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.106633324, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.035057724, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.026664659, 2024-04-25T13:15:02.544 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.035416665, 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1498, 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.056769884, 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000037897 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.545 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 436064 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:02.546 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 232, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 220, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1617556, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 127157, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.046631510, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.015008384, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.037322109, 2024-04-25T13:15:02.547 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.004322586, 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 219, 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.012345477, 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000056372 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 1616208, 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 126192 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T13:15:02.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3534, 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.090261660, 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000025540 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.550 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.551 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-25T13:15:02.552 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-25T13:15:02.553 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:02.553 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:02.553 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-04-25T13:15:02.553 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-25T13:15:02.553 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-04-25T13:15:02.554 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-04-25T13:15:02.555 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:02.556 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:02.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-04-25T13:15:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-04-25T13:15:02.560 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:02.561 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-25T13:15:02.562 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 4, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-04-25T13:15:02.563 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 2, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 4195424, 2024-04-25T13:15:02.564 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 4195283, 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 4 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-25T13:15:02.565 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 3, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 3, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 1, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 0, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 0, 2024-04-25T13:15:02.566 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 49652, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215416 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-04-25T13:15:02.567 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.568 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.569 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-25T13:15:02.570 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.572 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:02.573 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.575 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-25T13:15:02.575 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:02 smithi104 ceph-mon[32726]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T13:15:02.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:02 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:02.576 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.576 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.576 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.576 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-25T13:15:02.576 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.578 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.579 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.581 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.582 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.583 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.584 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.585 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.586 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.587 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 0, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-04-25T13:15:02.588 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 1198, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:02.589 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:02.590 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:02.591 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:02.592 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 197530, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 57, 2024-04-25T13:15:02.593 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 169, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 8512, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 13, 2024-04-25T13:15:02.595 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 2081, 2024-04-25T13:15:02.596 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 2081, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2081, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.056978931, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000507918 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-25T13:15:02.597 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2081, 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 2082 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 2081, 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 2081, 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-25T13:15:02.598 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 1343, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 736, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-25T13:15:02.599 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-04-25T13:15:02.600 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:02.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-25T13:15:02.602 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-25T13:15:02.603 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 97, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 11, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 6, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 6, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-04-25T13:15:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-25T13:15:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:02.606 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 2312, 2024-04-25T13:15:02.607 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1987852, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 2312, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2312, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1987852, 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:15:02.608 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T13:15:02.609 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 2081, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1125, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2081, 2024-04-25T13:15:02.610 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1125, 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:02.611 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 2081, 2024-04-25T13:15:02.612 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 2081, 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2081, 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 2081, 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:02.614 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.615 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x5613290626e0": { 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:02.616 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:02.617 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.618 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:02.618 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:15:02.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:02 smithi178 ceph-mon[33233]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T13:15:02.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:02 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:03.384 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T13:15:03.384 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T13:15:03.384 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T13:15:03.384 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-25T13:15:03.385 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-25T13:15:03.385 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:03.385 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:03.385 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T13:15:03.385 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T13:15:04.402 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:15:04.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:04 smithi038 ceph-mon[29400]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:15:04.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:04 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:04 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:04 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:04 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:04 smithi104 ceph-mon[32726]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:15:04.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:04 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:04 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:04 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:04 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:04 smithi178 ceph-mon[33233]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T13:15:04.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:04 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:04 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:04 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:04.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:04 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:05.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: Unable to set osd_memory_target on smithi178 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:05 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: Unable to set osd_memory_target on smithi178 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-04-25T13:15:05.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:05 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: Unable to set osd_memory_target on smithi178 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:05.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:05 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:06.114 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 218, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 207, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1588608, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 127264, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.045052499, 2024-04-25T13:15:06.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.013775602, 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.035365999, 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.004415673, 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 206, 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.011913010, 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000057830 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:06.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 1588608, 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 127264 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2, 2024-04-25T13:15:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 159, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 184064, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.014366613, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.010314496, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001674529, 2024-04-25T13:15:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:15:06.119 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:06.119 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 156, 2024-04-25T13:15:06.119 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.014081635, 2024-04-25T13:15:06.119 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000090266 2024-04-25T13:15:06.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 184064 2024-04-25T13:15:06.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 2992, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 448, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.000705339, 2024-04-25T13:15:06.121 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000045155, 2024-04-25T13:15:06.122 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000637944, 2024-04-25T13:15:06.122 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:15:06.122 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:06.122 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T13:15:06.122 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000754931, 2024-04-25T13:15:06.123 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000251643 2024-04-25T13:15:06.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.123 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:06.123 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:06.123 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:06.123 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.124 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:06.125 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:06.126 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:06.127 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:06.127 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:06.127 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:06.127 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:06.127 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:06.127 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:06.128 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:15:06.129 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 31, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:06.130 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-04-25T13:15:06.131 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T13:15:06.132 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 220, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1557883, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 220, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T13:15:06.133 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 220, 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1557883, 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:06.134 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T13:15:06.135 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T13:15:06.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:06 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-04-25T13:15:06.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:06 smithi038 ceph-mon[29400]: Unable to set osd_memory_target on smithi104 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T13:15:06.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:06 smithi038 ceph-mon[29400]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-04-25T13:15:06.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:06 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-04-25T13:15:06.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:06 smithi104 ceph-mon[32726]: Unable to set osd_memory_target on smithi104 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T13:15:06.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:06 smithi104 ceph-mon[32726]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-04-25T13:15:06.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:06 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-04-25T13:15:06.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:06 smithi178 ceph-mon[33233]: Unable to set osd_memory_target on smithi104 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T13:15:06.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:06 smithi178 ceph-mon[33233]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-04-25T13:15:06.818 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T13:15:06.818 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T13:15:06.818 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T13:15:06.818 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T13:15:06.819 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T13:15:06.819 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:15:06.819 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:15:06.819 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T13:15:06.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T13:15:07.069 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:15:07.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-25T13:15:07.548 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: Unable to set osd_memory_target on smithi038 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T13:15:07.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:07.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:07.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:07 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: Unable to set osd_memory_target on smithi038 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T13:15:07.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:07.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:07.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:07 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:07.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-25T13:15:07.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: Unable to set osd_memory_target on smithi038 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T13:15:07.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:07.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:07.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:07 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' 2024-04-25T13:15:09.154 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-25T13:15:09.155 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:09.155 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 969, 2024-04-25T13:15:09.155 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 754, 2024-04-25T13:15:09.155 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 262663, 2024-04-25T13:15:09.155 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 668633, 2024-04-25T13:15:09.155 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 20, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.105039454, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.031458332, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.037626676, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.030168886, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 752, 2024-04-25T13:15:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.288714846, 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000383929 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1020, 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000320155, 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000313 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 96768, 2024-04-25T13:15:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 344176 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2118, 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 796, 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 799689, 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 436060, 2024-04-25T13:15:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 12, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 8, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.260217332, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.067259709, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.090886199, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.093408693, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 794, 2024-04-25T13:15:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.175133542, 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000220571 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2364, 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000464819, 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000196 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 615344, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 243616 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1461, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 982, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2112586, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 381711, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2024-04-25T13:15:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.171023060, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.048242629, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.083528954, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.046128597, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 981, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.086976903, 2024-04-25T13:15:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000088661 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1649, 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000382421, 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000231 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1701616, 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 135712 2024-04-25T13:15:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 235, 2024-04-25T13:15:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.062363557, 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000265376 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000011552, 2024-04-25T13:15:09.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000005776 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 158, 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 158, 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 158, 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.129703789, 2024-04-25T13:15:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000820910 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 12, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-04-25T13:15:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 25, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 1, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 24, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 22, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 2, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 2, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 2, 2024-04-25T13:15:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 195, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 187, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-04-25T13:15:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-04-25T13:15:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 5, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 14, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 2, 2024-04-25T13:15:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 17 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 6, 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-04-25T13:15:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-04-25T13:15:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 17, 2024-04-25T13:15:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-04-25T13:15:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-04-25T13:15:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 106, 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-04-25T13:15:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 106, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 2, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-04-25T13:15:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-04-25T13:15:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-04-25T13:15:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-04-25T13:15:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4285058, 2024-04-25T13:15:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-04-25T13:15:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-04-25T13:15:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 84, 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.142151224, 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001692276 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 23, 2024-04-25T13:15:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 23, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 0, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 18, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 18, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 0, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 25, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 25, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 0, 2024-04-25T13:15:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 2, 2024-04-25T13:15:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 13, 2024-04-25T13:15:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 11, 2024-04-25T13:15:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 54320, 2024-04-25T13:15:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 231800 2024-04-25T13:15:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 200, 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 1, 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 158, 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 32, 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 2, 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4, 2024-04-25T13:15:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.004474070, 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001118517 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 109, 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.019975476, 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000183261 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000130612, 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000130612 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-04-25T13:15:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-04-25T13:15:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-04-25T13:15:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001645172, 2024-04-25T13:15:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000149561 2024-04-25T13:15:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6, 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.007186749, 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001197791 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.052415004, 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004765000 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-04-25T13:15:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000210368, 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000210368 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.004184812, 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002092406 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-04-25T13:15:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020089549, 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004017909 2024-04-25T13:15:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-04-25T13:15:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000866756, 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000866756 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-04-25T13:15:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7, 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018525221, 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002646460 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 5, 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 3, 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-04-25T13:15:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 27, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 13, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 612, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 375433, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 156, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 792, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 9, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-25T13:15:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 169, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 96002, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 602, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 645, 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 104917, 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 37, 2024-04-25T13:15:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 645, 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 645, 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 17.276796084, 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.026785730 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-04-25T13:15:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-04-25T13:15:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 645, 2024-04-25T13:15:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 676 2024-04-25T13:15:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 645, 2024-04-25T13:15:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 0, 2024-04-25T13:15:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 645, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 11, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 0, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 84, 2024-04-25T13:15:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 23, 2024-04-25T13:15:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-04-25T13:15:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-04-25T13:15:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-04-25T13:15:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-04-25T13:15:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 506, 2024-04-25T13:15:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 22, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-04-25T13:15:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-04-25T13:15:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-04-25T13:15:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-04-25T13:15:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-04-25T13:15:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-04-25T13:15:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 97, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 12, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-04-25T13:15:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 26, 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 4 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-04-25T13:15:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-04-25T13:15:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-04-25T13:15:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-04-25T13:15:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 4544, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 2671347, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 4544, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4544, 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2671347, 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 645, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 104917, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 645, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 104917, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 645, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 645, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 645, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 645, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55e4c8768500": { 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 60, 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 106, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 90754, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 106, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 84, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 90694, 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:09.236 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-25T13:15:10.048 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T13:15:10.048 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T13:15:10.049 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=6 2024-04-25T13:15:10.049 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T13:15:10.049 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T13:15:10.049 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-25T13:15:10.049 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-25T13:15:10.049 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T13:15:10.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T13:15:10.071 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:09 smithi104 ceph-mon[32726]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:15:10.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:09 smithi038 ceph-mon[29400]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:15:10.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:09 smithi178 ceph-mon[33233]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T13:15:10.293 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:15:12.037 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-25T13:15:12.038 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:12.038 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 981, 2024-04-25T13:15:12.038 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 966, 2024-04-25T13:15:12.038 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1885840, 2024-04-25T13:15:12.038 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 366006, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.135852007, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.051697915, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.055641523, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.029524865, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 965, 2024-04-25T13:15:12.039 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.051167897, 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000053023 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.040 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1636864, 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 130368 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 8, 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 163, 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 3868, 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 442938, 2024-04-25T13:15:12.041 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.010285435, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.006016864, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.002602557, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000055620, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 161, 2024-04-25T13:15:12.042 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012905623, 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000080159 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T13:15:12.043 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 442240 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1372, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1374, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 351300, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 436303, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-04-25T13:15:12.044 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.083150464, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.022186033, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.021758093, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.033747394, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1372, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.035731801, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000026043 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.045 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.046 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3596, 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.092134039, 2024-04-25T13:15:12.047 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000025621 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 0, 2024-04-25T13:15:12.050 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 1, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-04-25T13:15:12.051 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-04-25T13:15:12.052 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:12.053 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:12.054 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.055 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:12.056 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-04-25T13:15:12.057 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-04-25T13:15:12.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 4, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-04-25T13:15:12.059 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-04-25T13:15:12.060 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4195591, 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 4195200, 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 4 2024-04-25T13:15:12.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 3, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 3, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 0, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 1, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 1, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 0, 2024-04-25T13:15:12.062 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 0, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 0, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 0, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 52908, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215416 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.063 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.064 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:12.065 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.066 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.067 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.068 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.070 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.071 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.071 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-04-25T13:15:12.071 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.072 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.072 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.072 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-04-25T13:15:12.072 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.072 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-04-25T13:15:12.073 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.075 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.076 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.078 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:12.079 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.081 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.082 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.083 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 0, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 1734, 2024-04-25T13:15:12.084 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:12.085 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:12.086 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:12.087 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:12.088 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 181313, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 47, 2024-04-25T13:15:12.089 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:12.090 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 169, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 8512, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 13, 2024-04-25T13:15:12.091 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-04-25T13:15:12.092 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 2115, 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 2115, 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2115, 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.040987626, 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000492192 2024-04-25T13:15:12.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2115, 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2116 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 2115, 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 2115, 2024-04-25T13:15:12.094 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 1366, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 747, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-04-25T13:15:12.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2024-04-25T13:15:12.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:12.097 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-04-25T13:15:12.098 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-04-25T13:15:12.099 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 98, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 12, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 4, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 4, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-04-25T13:15:12.100 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:12.101 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:12.102 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:12.103 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 2357, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 2013613, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 2357, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:12.104 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2357, 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2013613, 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.105 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:12.106 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2115, 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 1292, 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2115, 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1292, 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:12.108 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2115, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 2115, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2115, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2115, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.109 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:12.110 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.111 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55e5bdb466e0": { 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:12.112 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:12.113 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:12.114 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:12.114 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:12.114 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.114 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-25T13:15:12.114 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:11 smithi104 ceph-mon[32726]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:15:12.114 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:11 smithi104 ceph-mon[32726]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:15:12.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:11 smithi178 ceph-mon[33233]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:15:12.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:11 smithi178 ceph-mon[33233]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:15:12.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:11 smithi038 ceph-mon[29400]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T13:15:12.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:11 smithi038 ceph-mon[29400]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:15:12.828 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T13:15:12.828 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T13:15:12.828 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T13:15:12.829 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T13:15:12.829 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T13:15:12.829 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:12.829 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:12.829 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T13:15:12.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T13:15:13.107 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:15:14.218 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:13 smithi104 ceph-mon[32726]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:15:14.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:13 smithi178 ceph-mon[33233]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:15:14.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:13 smithi038 ceph-mon[29400]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T13:15:14.795 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-25T13:15:14.795 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:14.795 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 125, 2024-04-25T13:15:14.795 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 129, 2024-04-25T13:15:14.795 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 47542, 2024-04-25T13:15:14.795 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 101410, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 15, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.013043263, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.003160829, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.008420792, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000787693, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 129, 2024-04-25T13:15:14.796 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.025570359, 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000198219 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 189, 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000031206, 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000165 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1840, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 3344 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 30, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 190, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 9862, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 460153, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 11, 2024-04-25T13:15:14.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.013984935, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.007017478, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.004637220, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000478632, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 186, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.023529406, 2024-04-25T13:15:14.799 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000126502 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18, 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000003192, 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000177 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 5488, 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 447184 2024-04-25T13:15:14.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 626, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 793, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1777680, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 276353, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.103698890, 2024-04-25T13:15:14.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.040936777, 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.061729184, 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.009002390, 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 792, 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.047497056, 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000059971 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 874, 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000227074, 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000259 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1648992, 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 133728 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-25T13:15:14.803 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 54, 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001652887, 2024-04-25T13:15:14.804 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000030609 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000007318, 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000003659 2024-04-25T13:15:14.805 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.806 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-04-25T13:15:14.806 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 7, 2024-04-25T13:15:14.806 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 5, 2024-04-25T13:15:14.806 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.002545818, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000509163 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 2, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:14.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 11, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 14, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 1, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 13, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 12, 2024-04-25T13:15:14.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 1, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 2, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 17, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 10, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 2, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 3, 2024-04-25T13:15:14.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-04-25T13:15:14.810 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-04-25T13:15:14.811 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:14.811 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:14.811 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:14.811 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 3, 2024-04-25T13:15:14.811 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 117, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:14.812 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-04-25T13:15:14.813 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 6, 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 4, 2024-04-25T13:15:14.814 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 6, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-04-25T13:15:14.815 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:14.816 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 6, 2024-04-25T13:15:14.817 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 6, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 2, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-04-25T13:15:14.818 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4195956, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-04-25T13:15:14.819 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.009585405, 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001917081 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 17, 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 17, 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 0, 2024-04-25T13:15:14.820 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 16, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 16, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 0, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 14, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 15, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 1, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 4, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 4, 2024-04-25T13:15:14.821 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 52464, 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223608 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 11, 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 7, 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 8, 2024-04-25T13:15:14.822 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 1, 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001751711, 2024-04-25T13:15:14.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000875855 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.824 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3, 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000794107, 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000264702 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.825 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.826 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.827 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-04-25T13:15:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.829 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.829 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.829 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-04-25T13:15:14.829 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.830 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.830 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.830 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-04-25T13:15:14.830 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:14.833 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.835 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.836 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-04-25T13:15:14.839 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 0, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 0, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 4, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 2, 2024-04-25T13:15:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 85, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:14.846 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 263881, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 94, 2024-04-25T13:15:14.847 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8248, 2024-04-25T13:15:14.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 172, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 69104, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 261, 2024-04-25T13:15:14.849 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-04-25T13:15:14.850 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 43, 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 12099, 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 28, 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 43, 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 43, 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.243997125, 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005674351 2024-04-25T13:15:14.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 43, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 67 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 43, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 0, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 43, 2024-04-25T13:15:14.852 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 0, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 5, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 7, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-04-25T13:15:14.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 6, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-04-25T13:15:14.854 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:14.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-04-25T13:15:14.856 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-04-25T13:15:14.857 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 98, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 12, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 8, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 8, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 17, 2024-04-25T13:15:14.858 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 2 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-04-25T13:15:14.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-04-25T13:15:14.860 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:14.861 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 777, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1747996, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 777, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 777, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1747996, 2024-04-25T13:15:14.862 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:14.863 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 43, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 12099, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 43, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 12099, 2024-04-25T13:15:14.864 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:14.865 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 43, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 43, 2024-04-25T13:15:14.866 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 43, 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 43, 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:14.867 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:14.868 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:14.869 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55974c5e0500": { 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 6, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1652, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 6, 2024-04-25T13:15:14.870 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 5, 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1652, 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.871 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:14.872 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=6 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:15.711 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:15.711 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T13:15:15.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T13:15:15.960 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:15:16.075 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:15 smithi104 ceph-mon[32726]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:15:16.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:15 smithi178 ceph-mon[33233]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:15:16.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:15 smithi038 ceph-mon[29400]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:15:17.211 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:16 smithi104 ceph-mon[32726]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:15:17.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:16 smithi178 ceph-mon[33233]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:15:17.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:16 smithi038 ceph-mon[29400]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:15:17.624 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-25T13:15:17.624 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 752, 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 754, 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 199267, 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 239200, 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 2, 2024-04-25T13:15:17.625 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.070071961, 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.028547976, 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.016061790, 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.017821439, 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 752, 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.030135674, 2024-04-25T13:15:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000040074 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:15:17.627 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 636, 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 793, 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 158314, 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 642937, 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-04-25T13:15:17.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.066763312, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.024673020, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.014164798, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.020543679, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 791, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.035765750, 2024-04-25T13:15:17.629 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000045215 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T13:15:17.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 442592 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 991, 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 976, 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1889072, 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 370332, 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 3, 2024-04-25T13:15:17.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 2, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.136288952, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.042149697, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.060055131, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.034669157, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 975, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.058516653, 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000060017 2024-04-25T13:15:17.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1638416, 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 133104 2024-04-25T13:15:17.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:17.634 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3572, 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.095549619, 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000026749 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-25T13:15:17.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-04-25T13:15:17.636 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-04-25T13:15:17.637 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:17.638 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-04-25T13:15:17.639 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 1, 2024-04-25T13:15:17.639 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-04-25T13:15:17.639 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-04-25T13:15:17.639 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-04-25T13:15:17.639 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-04-25T13:15:17.639 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-04-25T13:15:17.640 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2024-04-25T13:15:17.641 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:17.642 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:17.643 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-04-25T13:15:17.644 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:17.644 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:17.644 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-04-25T13:15:17.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.644 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:17.645 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:17.646 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-04-25T13:15:17.647 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-04-25T13:15:17.648 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 6, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-04-25T13:15:17.649 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4195956, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 4195815, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 6 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 3, 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 3, 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 0, 2024-04-25T13:15:17.651 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 1, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 1, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-04-25T13:15:17.652 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 49224, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215416 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-04-25T13:15:17.653 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.656 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-04-25T13:15:17.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.658 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.660 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-04-25T13:15:17.660 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.660 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.661 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.662 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.663 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.664 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.666 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.667 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-04-25T13:15:17.668 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.670 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 0, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-04-25T13:15:17.672 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 1201, 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:17.673 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:17.674 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:17.675 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:17.676 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 189463, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 52, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:17.677 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:17.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 169, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 8512, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 13, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-25T13:15:17.679 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 2128, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2024-04-25T13:15:17.680 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 2128, 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2128, 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.092987023, 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000513621 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2128, 2024-04-25T13:15:17.681 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2129 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 2128, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 2128, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 1374, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-04-25T13:15:17.682 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 752, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:17.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-04-25T13:15:17.684 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-04-25T13:15:17.685 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-04-25T13:15:17.686 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 99, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 13, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 5, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 5, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-04-25T13:15:17.687 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-04-25T13:15:17.688 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-04-25T13:15:17.689 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 2375, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 2017388, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 2375, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2375, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2017388, 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:17.692 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2128, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 1657, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2128, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1657, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.693 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:17.694 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2128, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 2128, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2128, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2128, 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:17.695 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:17.696 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.697 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55e954c4a6e0": { 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-25T13:15:17.698 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-25T13:15:17.699 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-25T13:15:17.700 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.700 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-25T13:15:17.700 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-25T13:15:18.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:17 smithi178 ceph-mon[33233]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:15:18.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:17 smithi038 ceph-mon[29400]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:15:18.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:17 smithi104 ceph-mon[32726]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:15:18.448 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T13:15:18.449 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T13:15:18.449 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T13:15:18.449 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-25T13:15:18.449 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-25T13:15:18.449 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:18.449 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:18.449 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T13:15:18.450 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T13:15:18.698 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:15:20.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:19 smithi178 ceph-mon[33233]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:15:20.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:19 smithi038 ceph-mon[29400]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:15:20.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:19 smithi104 ceph-mon[32726]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-25T13:15:20.458 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-25T13:15:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 311, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 500, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 102661, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 96875, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 9, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.050517585, 2024-04-25T13:15:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.019028356, 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.023218016, 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.004793258, 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 500, 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.070948412, 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000141896 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 764, 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000236955, 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000310 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 112, 2024-04-25T13:15:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 263, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 46473, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 489397, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 10, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.022940215, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.010084051, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.011790965, 2024-04-25T13:15:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000279902, 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 265, 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.032366277, 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000122136 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 171, 2024-04-25T13:15:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000040622, 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000237 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 443856 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 330, 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 320, 2024-04-25T13:15:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1667715, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 212003, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 12, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 7, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.055360001, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.016712336, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.043568016, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005731246, 2024-04-25T13:15:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 313, 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.022107718, 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000070631 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 88, 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000018166, 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000206 2024-04-25T13:15:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1645600, 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 135984 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-04-25T13:15:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 40, 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001771466, 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000044286 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000009549, 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000004774 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-04-25T13:15:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 13, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 13, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 12, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-04-25T13:15:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 2, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 1, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 1, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 3, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 117, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-04-25T13:15:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 3, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 2, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 3, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-04-25T13:15:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-04-25T13:15:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-04-25T13:15:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 2, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 2, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-04-25T13:15:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-04-25T13:15:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4194892, 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001547535, 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001547535 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-04-25T13:15:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 16, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 16, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 15, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 16, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 1, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 13, 2024-04-25T13:15:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 13, 2024-04-25T13:15:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-04-25T13:15:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-04-25T13:15:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-04-25T13:15:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-04-25T13:15:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 52440, 2024-04-25T13:15:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223608 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-04-25T13:15:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-04-25T13:15:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-04-25T13:15:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-04-25T13:15:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.503 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 0, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 0, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 0, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-04-25T13:15:20.505 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 0, 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:20.506 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:20.507 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:20.508 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:20.509 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 222009, 2024-04-25T13:15:20.510 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 69, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:20.511 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 169, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 64832, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 238, 2024-04-25T13:15:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-04-25T13:15:20.513 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 35, 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 10710, 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 35, 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 35, 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.198996371, 2024-04-25T13:15:20.514 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.005685610 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 35, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 57 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 35, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 0, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 35, 2024-04-25T13:15:20.515 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 0, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 1, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 5, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-04-25T13:15:20.516 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 6, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-04-25T13:15:20.517 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:20.518 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-04-25T13:15:20.519 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-04-25T13:15:20.520 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 99, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 13, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 8, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 8, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-04-25T13:15:20.521 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 14, 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 1 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-04-25T13:15:20.522 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:20.523 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:20.524 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 749, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1732661, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 749, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:20.525 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 749, 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1732661, 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.526 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:20.527 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 35, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 10710, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 35, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 10710, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.528 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:20.529 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 35, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 35, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 35, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 35, 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:20.530 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-04-25T13:15:20.532 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x563900ffa500": { 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:20.533 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 2, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 588, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 2, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:20.534 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1, 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 588, 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:20.535 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-25T13:15:21.213 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T13:15:21.214 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T13:15:21.214 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T13:15:21.214 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=3 2024-04-25T13:15:21.214 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T13:15:21.214 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:21.214 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:21.214 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T13:15:21.215 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T13:15:21.463 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:15:22.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:21 smithi178 ceph-mon[33233]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-25T13:15:22.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:21 smithi038 ceph-mon[29400]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-25T13:15:22.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:21 smithi104 ceph-mon[32726]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-25T13:15:23.142 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-25T13:15:23.142 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:23.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1547, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1549, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 551761, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 491956, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.154700156, 2024-04-25T13:15:23.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.049256151, 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.038874162, 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.051522969, 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1547, 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.063474677, 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000041030 2024-04-25T13:15:23.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:15:23.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 6, 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 164, 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1976, 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 444922, 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-04-25T13:15:23.146 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.010828203, 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.006424105, 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.002415297, 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000096483, 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 161, 2024-04-25T13:15:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.020765333, 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000128977 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.148 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 443584 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 991, 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 974, 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1823873, 2024-04-25T13:15:23.149 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 368707, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.126017059, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.039666017, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.056511761, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.033507824, 2024-04-25T13:15:23.150 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 974, 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.036527556, 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000037502 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1641024, 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 132432 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-04-25T13:15:23.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3699, 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.114597417, 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000030980 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.154 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.155 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-04-25T13:15:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 12, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 12, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 12, 2024-04-25T13:15:23.157 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-04-25T13:15:23.158 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-04-25T13:15:23.159 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-04-25T13:15:23.160 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:23.161 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-04-25T13:15:23.162 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-04-25T13:15:23.163 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-04-25T13:15:23.164 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-04-25T13:15:23.165 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:23.166 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 107, 2024-04-25T13:15:23.167 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-04-25T13:15:23.168 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4285170, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 4285058, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.169 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 107 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 13, 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 13, 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 8, 2024-04-25T13:15:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 8, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 12, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 12, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-04-25T13:15:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 49852, 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215416 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-04-25T13:15:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-04-25T13:15:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 5, 2024-04-25T13:15:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 3, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 4, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2799, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-04-25T13:15:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-04-25T13:15:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189423, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 51, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 169, 2024-04-25T13:15:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 47746, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 220, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-04-25T13:15:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 2294, 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 26, 2024-04-25T13:15:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 4, 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 2294, 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2294, 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.034980573, 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000451168 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2294, 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2295 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 2294, 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 2294, 2024-04-25T13:15:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 1494, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 797, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-04-25T13:15:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-04-25T13:15:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-04-25T13:15:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-04-25T13:15:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 99, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-04-25T13:15:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 13, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-04-25T13:15:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-04-25T13:15:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 2540, 2024-04-25T13:15:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 2133553, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 2540, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2540, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2133553, 2024-04-25T13:15:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2294, 2024-04-25T13:15:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 90876, 2024-04-25T13:15:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2294, 2024-04-25T13:15:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 90876, 2024-04-25T13:15:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-04-25T13:15:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2294, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 2294, 2024-04-25T13:15:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2294, 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2294, 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55c4d5a286e0": { 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:23.226 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:23.226 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:23.226 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.226 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:23.226 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-25T13:15:23.834 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T13:15:23.834 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T13:15:23.834 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T13:15:23.834 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T13:15:23.834 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-25T13:15:23.835 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:23.835 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:23.835 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T13:15:23.835 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T13:15:24.077 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:15:24.198 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:23 smithi178 ceph-mon[33233]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:15:24.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:23 smithi038 ceph-mon[29400]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:15:24.324 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:23 smithi104 ceph-mon[32726]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T13:15:25.645 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-25T13:15:25.645 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:25.645 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 385, 2024-04-25T13:15:25.645 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 496, 2024-04-25T13:15:25.645 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1698189, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 547082, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 15, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.064126539, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.023506294, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.041518746, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005629101, 2024-04-25T13:15:25.646 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 492, 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.040895604, 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000083121 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 179, 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000049222, 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000274 2024-04-25T13:15:25.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1654128, 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 452320 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 350, 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 613, 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 119086, 2024-04-25T13:15:25.648 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 451074, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 11, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.065806442, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.026201060, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.031632929, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.004954957, 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:25.649 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 614, 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.042688780, 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000069525 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 885, 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000283568, 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000320 2024-04-25T13:15:25.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 8768, 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 146304 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 61, 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 63, 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 23806, 2024-04-25T13:15:25.651 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 31990, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 8, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.006801131, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.001771775, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.005223459, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000305874, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 61, 2024-04-25T13:15:25.652 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.026636878, 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000436670 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 93, 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000026894, 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000289 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.653 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-04-25T13:15:25.654 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 37, 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001967420, 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000053173 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:25.655 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000013431, 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000006715 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-04-25T13:15:25.656 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 7, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 3, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000714262, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000238087 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 4, 2024-04-25T13:15:25.657 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 14, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 1, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 13, 2024-04-25T13:15:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 12, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 1, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 1, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 12, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 8, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 1, 2024-04-25T13:15:25.659 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 2, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 1000, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 1, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-04-25T13:15:25.660 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 3, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 117, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:25.661 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:25.662 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:25.663 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 6, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 4, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 5, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-04-25T13:15:25.664 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:25.665 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-04-25T13:15:25.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 4, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-04-25T13:15:25.667 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 4, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-04-25T13:15:25.668 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4195591, 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.004588406, 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001529468 2024-04-25T13:15:25.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 17, 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 17, 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 16, 2024-04-25T13:15:25.670 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 17, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 1, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 14, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 14, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 1, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 1, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 52192, 2024-04-25T13:15:25.671 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223608 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 7, 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 7, 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 18, 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-04-25T13:15:25.672 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000714262, 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000238087 2024-04-25T13:15:25.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.674 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.675 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.677 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.678 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-04-25T13:15:25.679 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.679 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.679 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.679 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-04-25T13:15:25.679 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.680 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.680 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.680 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-04-25T13:15:25.680 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.680 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-04-25T13:15:25.681 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.682 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.683 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.684 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.686 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.687 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.688 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.689 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.690 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.691 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 2, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 2, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 1, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 628, 2024-04-25T13:15:25.692 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:25.693 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:25.694 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:25.695 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:25.696 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 256009, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 97, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 176, 2024-04-25T13:15:25.697 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 2, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:25.698 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 169, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 69424, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 257, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:25.699 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 38, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 11499, 2024-04-25T13:15:25.700 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 28, 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 38, 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 38, 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.231995757, 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.006105151 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-04-25T13:15:25.701 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 38, 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 60 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 38, 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 0, 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 38, 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-04-25T13:15:25.702 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 0, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 3, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 6, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-04-25T13:15:25.703 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 6, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:25.704 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-04-25T13:15:25.705 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-04-25T13:15:25.706 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 99, 2024-04-25T13:15:25.707 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 13, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 9, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 9, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 14, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 1 2024-04-25T13:15:25.708 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-04-25T13:15:25.709 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-04-25T13:15:25.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 792, 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1751966, 2024-04-25T13:15:25.711 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 792, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 792, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1751966, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.712 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:25.713 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 38, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 11499, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 38, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 11499, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.714 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:25.715 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 38, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 38, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 38, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 38, 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:25.716 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:25.717 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-04-25T13:15:25.718 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55a202c1e500": { 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:25.719 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 4, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1287, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 4, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 3, 2024-04-25T13:15:25.720 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1287, 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:25.721 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-25T13:15:26.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:25 smithi178 ceph-mon[33233]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:15:26.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:25 smithi038 ceph-mon[29400]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:15:26.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:25 smithi104 ceph-mon[32726]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T13:15:26.361 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T13:15:26.361 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T13:15:26.362 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T13:15:26.362 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=6 2024-04-25T13:15:26.362 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-25T13:15:26.362 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:26.362 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:26.362 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T13:15:26.363 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d3627c2a-0302-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T13:15:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:15:27.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:27 smithi178 ceph-mon[33233]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 94 B/s wr, 9 op/s 2024-04-25T13:15:28.240 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-25T13:15:28.241 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:15:28.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1516, 2024-04-25T13:15:28.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1518, 2024-04-25T13:15:28.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 453801, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 482089, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.159913508, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.054804168, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.039757738, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.050190156, 2024-04-25T13:15:28.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:28.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1516, 2024-04-25T13:15:28.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.060288609, 2024-04-25T13:15:28.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000039768 2024-04-25T13:15:28.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.243 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.244 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 251, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 236, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1640622, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 136046, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.049095493, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.015041737, 2024-04-25T13:15:28.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.038888430, 2024-04-25T13:15:28.246 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005518133, 2024-04-25T13:15:28.246 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:28.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 235, 2024-04-25T13:15:28.246 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.014412147, 2024-04-25T13:15:28.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000061328 2024-04-25T13:15:28.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1639232, 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 135024 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.247 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 7, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 167, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3580, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 446670, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.010533743, 2024-04-25T13:15:28.248 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.006349381, 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.002284569, 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000024283, 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 165, 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.015551031, 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000094248 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.249 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 446352 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.250 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:28.251 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3033, 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.090773356, 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000029928 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-04-25T13:15:28.252 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-04-25T13:15:28.253 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-04-25T13:15:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-04-25T13:15:28.255 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-04-25T13:15:28.256 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-04-25T13:15:28.257 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-04-25T13:15:28.258 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:15:28.259 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:15:28.260 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-04-25T13:15:28.261 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-04-25T13:15:28.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:15:28.263 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-04-25T13:15:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 2, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-04-25T13:15:28.265 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-04-25T13:15:28.266 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4194892, 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 4194352, 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 2 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 3, 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 3, 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 1, 2024-04-25T13:15:28.268 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 1, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 50288, 2024-04-25T13:15:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215416 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-04-25T13:15:28.270 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.271 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.272 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.273 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.273 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-04-25T13:15:28.273 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.273 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-04-25T13:15:28.275 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.276 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.276 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.276 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-04-25T13:15:28.277 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.277 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.277 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.277 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-04-25T13:15:28.277 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.278 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.278 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.278 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-04-25T13:15:28.278 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.278 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.279 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.279 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-04-25T13:15:28.279 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.281 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.283 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.284 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.286 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.288 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.290 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.291 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 0, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 0, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 0, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-04-25T13:15:28.292 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 0, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:15:28.293 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:15:28.294 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:15:28.295 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-04-25T13:15:28.296 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189393, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 51, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-04-25T13:15:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8128, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 169, 2024-04-25T13:15:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 8512, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 13, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-04-25T13:15:28.299 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 1520, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 1520, 2024-04-25T13:15:28.300 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1520, 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.796985029, 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000524332 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-04-25T13:15:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1520, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1521 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 1520, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 1520, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-04-25T13:15:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 757, 2024-04-25T13:15:28.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-04-25T13:15:28.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 761, 2024-04-25T13:15:28.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-04-25T13:15:28.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-04-25T13:15:28.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-04-25T13:15:28.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-04-25T13:15:28.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:15:28.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-04-25T13:15:28.306 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-04-25T13:15:28.307 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 99, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 13, 2024-04-25T13:15:28.308 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-04-25T13:15:28.309 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-04-25T13:15:28.309 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-04-25T13:15:28.309 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-04-25T13:15:28.309 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-04-25T13:15:28.309 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-04-25T13:15:28.309 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-04-25T13:15:28.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.310 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-04-25T13:15:28.310 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-04-25T13:15:28.310 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:15:28.310 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-04-25T13:15:28.310 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-04-25T13:15:28.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.311 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-04-25T13:15:28.311 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-04-25T13:15:28.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.311 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-04-25T13:15:28.311 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-04-25T13:15:28.312 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:15:28.312 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-04-25T13:15:28.312 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-04-25T13:15:28.312 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-04-25T13:15:28.312 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-04-25T13:15:28.312 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-04-25T13:15:28.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.313 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:15:28.313 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:28.313 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:28.313 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1770, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1923032, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1770, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:28.314 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1770, 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1923032, 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1520, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 593, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1520, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 593, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.317 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:28.318 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1520, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1520, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1520, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1520, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:28.320 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x5654cdfc06e0": { 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-04-25T13:15:28.322 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-04-25T13:15:28.323 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.324 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-25T13:15:28.324 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-25T13:15:28.324 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:27 smithi038 ceph-mon[29400]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 94 B/s wr, 9 op/s 2024-04-25T13:15:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:27 smithi104 ceph-mon[32726]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 94 B/s wr, 9 op/s 2024-04-25T13:15:29.069 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T13:15:29.069 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T13:15:29.069 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T13:15:29.069 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-25T13:15:29.069 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-25T13:15:29.069 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:15:29.070 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:15:29.070 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-04-25T13:15:29.079 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T13:15:29.089 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:15:29.090 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T13:15:29.090 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:15:29.115 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T13:15:29.115 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:15:29.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:15:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 -- 172.21.15.38:0/3892206655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 msgr2=0x7f76a810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 --2- 172.21.15.38:0/3892206655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a810dd40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009900 tx=0x7f76a002a410 comp rx=0 tx=0).stop 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 -- 172.21.15.38:0/3892206655 shutdown_connections 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 --2- 172.21.15.38:0/3892206655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a810dd40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 --2- 172.21.15.38:0/3892206655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a8101a50 0x7f76a8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 --2- 172.21.15.38:0/3892206655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a81010a0 0x7f76a8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 -- 172.21.15.38:0/3892206655 >> 172.21.15.38:0/3892206655 conn(0x7f76a8077f50 msgr2=0x7f76a8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 -- 172.21.15.38:0/3892206655 shutdown_connections 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.330+0000 7f76aebed640 1 -- 172.21.15.38:0/3892206655 wait complete. 2024-04-25T13:15:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 Processor -- start 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 -- start start 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a81010a0 0x7f76a81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a819bf60 0x7f76a819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8111210 con 0x7f76a8106170 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8111090 con 0x7f76a819bf60 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76aebed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8111390 con 0x7f76a81010a0 2024-04-25T13:15:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a8197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39726/0 (socket says 172.21.15.38:39726) 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 -- 172.21.15.38:0/1194183354 learned_addr learned my addr 172.21.15.38:0/1194183354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76ac989640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a81010a0 0x7f76a81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76ad18a640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a819bf60 0x7f76a819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 -- 172.21.15.38:0/1194183354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a81010a0 msgr2=0x7f76a81976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a81010a0 0x7f76a81976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 -- 172.21.15.38:0/1194183354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a819bf60 msgr2=0x7f76a819c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a819bf60 0x7f76a819c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76a7fff640 1 -- 172.21.15.38:0/1194183354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a819ca50 con 0x7f76a8106170 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.331+0000 7f76ad18a640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a819bf60 0x7f76a819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.332+0000 7f76a7fff640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a8197c30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f769800ebc0 tx=0x7f769800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.332+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698004d20 con 0x7f76a8106170 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.332+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f76980054f0 con 0x7f76a8106170 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.332+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a819cd40 con 0x7f76a8106170 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.332+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698010930 con 0x7f76a8106170 2024-04-25T13:15:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.332+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a44e0 con 0x7f76a8106170 2024-04-25T13:15:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.333+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8068d70 con 0x7f76a8106170 2024-04-25T13:15:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.334+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f76980056a0 con 0x7f76a8106170 2024-04-25T13:15:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.335+0000 7f76a5ffb640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f76940a3d20 0x7f76940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.335+0000 7f76ac989640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f76940a3d20 0x7f76940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.335+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f76980d10f0 con 0x7f76a8106170 2024-04-25T13:15:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.336+0000 7f76ac989640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f76940a3d20 0x7f76940a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7690005d00 tx=0x7f769000b410 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:29.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.340+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f769809d720 con 0x7f76a8106170 2024-04-25T13:15:29.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.528+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a8109460 con 0x7f76a8106170 2024-04-25T13:15:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.529+0000 7f76a5ffb640 1 -- 172.21.15.38:0/1194183354 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23117 (secure 0 0 0) 0x7f769809d020 con 0x7f76a8106170 2024-04-25T13:15:29.530 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:15:29.530 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":99,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:15:15.878896+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/448455346":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1381358988":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/3705953772":"2024-04-26T13:04:48.566935+0000","172.21.15.38:6801/1333793113":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1475454542":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/1333793113":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:15:29.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f76940a3d20 msgr2=0x7f76940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f76940a3d20 0x7f76940a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7690005d00 tx=0x7f769000b410 comp rx=0 tx=0).stop 2024-04-25T13:15:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 msgr2=0x7f76a8197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a8197c30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f769800ebc0 tx=0x7f769800c6a0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 shutdown_connections 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76a819bf60 0x7f76a819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8106170 0x7f76a8197c30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f76940a3d20 0x7f76940a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 --2- 172.21.15.38:0/1194183354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a81010a0 0x7f76a81976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 >> 172.21.15.38:0/1194183354 conn(0x7f76a8077f50 msgr2=0x7f76a80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 shutdown_connections 2024-04-25T13:15:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.532+0000 7f76aebed640 1 -- 172.21.15.38:0/1194183354 wait complete. 2024-04-25T13:15:29.543 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:58:01.286924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:02:42.235731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:02:43.206179+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:15:29.543 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:15:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 -- 172.21.15.38:0/3745998480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d4106160 msgr2=0x7fc8d410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 --2- 172.21.15.38:0/3745998480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d4106160 0x7fc8d410dd30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009ca0 tx=0x7fc8cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 -- 172.21.15.38:0/3745998480 shutdown_connections 2024-04-25T13:15:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 --2- 172.21.15.38:0/3745998480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d4106160 0x7fc8d410dd30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 --2- 172.21.15.38:0/3745998480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8d4101a40 0x7fc8d4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 --2- 172.21.15.38:0/3745998480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8d4101090 0x7fc8d4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 -- 172.21.15.38:0/3745998480 >> 172.21.15.38:0/3745998480 conn(0x7fc8d4077f50 msgr2=0x7fc8d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 -- 172.21.15.38:0/3745998480 shutdown_connections 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 -- 172.21.15.38:0/3745998480 wait complete. 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.649+0000 7fc8daa69640 1 Processor -- start 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 -- start start 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8d4101090 0x7fc8d4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8d4106160 0x7fc8d4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 0x7fc8d41a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41111e0 con 0x7fc8d419a120 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4111060 con 0x7fc8d4106160 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8daa69640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4111360 con 0x7fc8d4101090 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 0x7fc8d41a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 0x7fc8d41a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39732/0 (socket says 172.21.15.38:39732) 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 -- 172.21.15.38:0/4169303696 learned_addr learned my addr 172.21.15.38:0/4169303696 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d8805640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8d4101090 0x7fc8d4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d3fff640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8d4106160 0x7fc8d4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 -- 172.21.15.38:0/4169303696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8d4101090 msgr2=0x7fc8d4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8d4101090 0x7fc8d4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 -- 172.21.15.38:0/4169303696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8d4106160 msgr2=0x7fc8d4199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8d4106160 0x7fc8d4199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 -- 172.21.15.38:0/4169303696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d41a4ea0 con 0x7fc8d419a120 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.650+0000 7fc8d9006640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 0x7fc8d41a4680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc005e30 tx=0x7fc8cc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.651+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc031650 con 0x7fc8d419a120 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.651+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d41a5130 con 0x7fc8d419a120 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.651+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc8cc0317f0 con 0x7fc8d419a120 2024-04-25T13:15:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.651+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc02fb70 con 0x7fc8d419a120 2024-04-25T13:15:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.651+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41a5680 con 0x7fc8d419a120 2024-04-25T13:15:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.652+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d4068d70 con 0x7fc8d419a120 2024-04-25T13:15:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.653+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8cc02fd70 con 0x7fc8d419a120 2024-04-25T13:15:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.654+0000 7fc8d1ffb640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc8bc0a3d20 0x7fc8bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.654+0000 7fc8d8805640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc8bc0a3d20 0x7fc8bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.654+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc8cc0f1bb0 con 0x7fc8d419a120 2024-04-25T13:15:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.654+0000 7fc8d8805640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc8bc0a3d20 0x7fc8bc0a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005d00 tx=0x7fc8c400b410 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:29.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.658+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8cc0be2f0 con 0x7fc8d419a120 2024-04-25T13:15:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.846+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8d4101a40 con 0x7fc8d419a120 2024-04-25T13:15:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.847+0000 7fc8d1ffb640 1 -- 172.21.15.38:0/4169303696 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fc8cc0bdbf0 con 0x7fc8d419a120 2024-04-25T13:15:29.847 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc8bc0a3d20 msgr2=0x7fc8bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc8bc0a3d20 0x7fc8bc0a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005d00 tx=0x7fc8c400b410 comp rx=0 tx=0).stop 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 msgr2=0x7fc8d41a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 0x7fc8d41a4680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc005e30 tx=0x7fc8cc003f90 comp rx=0 tx=0).stop 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 shutdown_connections 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8d419a120 0x7fc8d41a4680 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8d4106160 0x7fc8d4199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc8bc0a3d20 0x7fc8bc0a61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 --2- 172.21.15.38:0/4169303696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8d4101090 0x7fc8d4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 >> 172.21.15.38:0/4169303696 conn(0x7fc8d4077f50 msgr2=0x7fc8d40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 shutdown_connections 2024-04-25T13:15:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.849+0000 7fc8daa69640 1 -- 172.21.15.38:0/4169303696 wait complete. 2024-04-25T13:15:29.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:15:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 -- 172.21.15.38:0/1361620553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c107270 msgr2=0x7fcc7c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 --2- 172.21.15.38:0/1361620553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c107270 0x7fcc7c10ee40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009ca0 tx=0x7fcc7402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 -- 172.21.15.38:0/1361620553 shutdown_connections 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 --2- 172.21.15.38:0/1361620553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c107270 0x7fcc7c10ee40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 --2- 172.21.15.38:0/1361620553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7c102b50 0x7fcc7c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 --2- 172.21.15.38:0/1361620553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 -- 172.21.15.38:0/1361620553 >> 172.21.15.38:0/1361620553 conn(0x7fcc7c0fda40 msgr2=0x7fcc7c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 -- 172.21.15.38:0/1361620553 shutdown_connections 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.965+0000 7fcc81a31640 1 -- 172.21.15.38:0/1361620553 wait complete. 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 Processor -- start 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 -- start start 2024-04-25T13:15:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c107270 0x7fcc7c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7c1986b0 0x7fcc7c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c1122f0 con 0x7fcc7c1021a0 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c112170 con 0x7fcc7c1986b0 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc81a31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c112470 con 0x7fcc7c107270 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56580/0 (socket says 172.21.15.38:56580) 2024-04-25T13:15:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc7affd640 1 -- 172.21.15.38:0/693346205 learned_addr learned my addr 172.21.15.38:0/693346205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc73fff640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c107270 0x7fcc7c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.966+0000 7fcc7b7fe640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7c1986b0 0x7fcc7c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc7affd640 1 -- 172.21.15.38:0/693346205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c107270 msgr2=0x7fcc7c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc7affd640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c107270 0x7fcc7c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc7affd640 1 -- 172.21.15.38:0/693346205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7c1986b0 msgr2=0x7fcc7c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc7affd640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7c1986b0 0x7fcc7c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc7affd640 1 -- 172.21.15.38:0/693346205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c1992b0 con 0x7fcc7c1021a0 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc73fff640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c107270 0x7fcc7c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:15:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc7affd640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c19e5a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d950 tx=0x7fcc6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc7c1021a0 2024-04-25T13:15:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcc6c009c30 con 0x7fcc7c1021a0 2024-04-25T13:15:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c1a5340 con 0x7fcc7c1021a0 2024-04-25T13:15:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00f6b0 con 0x7fcc7c1021a0 2024-04-25T13:15:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.967+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c1a5740 con 0x7fcc7c1021a0 2024-04-25T13:15:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.968+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c103670 con 0x7fcc7c1021a0 2024-04-25T13:15:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.969+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc6c0109c0 con 0x7fcc7c1021a0 2024-04-25T13:15:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.970+0000 7fcc80a2f640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcc680a3d20 0x7fcc680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.970+0000 7fcc73fff640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcc680a3d20 0x7fcc680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.970+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fcc6c0d0510 con 0x7fcc7c1021a0 2024-04-25T13:15:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.970+0000 7fcc73fff640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcc680a3d20 0x7fcc680a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005fd0 tx=0x7fcc64005850 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:29.973+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc6c009de0 con 0x7fcc7c1021a0 2024-04-25T13:15:30.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.158+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcc7c0691c0 con 0x7fcc7c1021a0 2024-04-25T13:15:30.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.158+0000 7fcc80a2f640 1 -- 172.21.15.38:0/693346205 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fcc6c09cba0 con 0x7fcc7c1021a0 2024-04-25T13:15:30.158 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T13:15:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcc680a3d20 msgr2=0x7fcc680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcc680a3d20 0x7fcc680a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005fd0 tx=0x7fcc64005850 comp rx=0 tx=0).stop 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 msgr2=0x7fcc7c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c19e5a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d950 tx=0x7fcc6c00de10 comp rx=0 tx=0).stop 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 shutdown_connections 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7c1986b0 0x7fcc7c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcc680a3d20 0x7fcc680a61e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7c107270 0x7fcc7c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 --2- 172.21.15.38:0/693346205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc7c1021a0 0x7fcc7c19e5a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 >> 172.21.15.38:0/693346205 conn(0x7fcc7c0fda40 msgr2=0x7fcc7c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:30.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 shutdown_connections 2024-04-25T13:15:30.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.161+0000 7fcc81a31640 1 -- 172.21.15.38:0/693346205 wait complete. 2024-04-25T13:15:30.170 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:29 smithi038 ceph-mon[29400]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:15:30.170 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:29 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1194183354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:15:30.170 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:29 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/4169303696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:15:30.171 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:15:30.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:29 smithi178 ceph-mon[33233]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:15:30.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:29 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1194183354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:15:30.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:29 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/4169303696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:15:30.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 -- 172.21.15.38:0/499937031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102b40 msgr2=0x7fcb70106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 --2- 172.21.15.38:0/499937031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102b40 0x7fcb70106b80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009900 tx=0x7fcb6402a410 comp rx=0 tx=0).stop 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 -- 172.21.15.38:0/499937031 shutdown_connections 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 --2- 172.21.15.38:0/499937031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70107260 0x7fcb7010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 --2- 172.21.15.38:0/499937031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102b40 0x7fcb70106b80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 --2- 172.21.15.38:0/499937031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb70102190 0x7fcb70102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 -- 172.21.15.38:0/499937031 >> 172.21.15.38:0/499937031 conn(0x7fcb700793a0 msgr2=0x7fcb700797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 -- 172.21.15.38:0/499937031 shutdown_connections 2024-04-25T13:15:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.276+0000 7fcb76320640 1 -- 172.21.15.38:0/499937031 wait complete. 2024-04-25T13:15:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 Processor -- start 2024-04-25T13:15:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 -- start start 2024-04-25T13:15:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 0x7fcb7019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb70107260 0x7fcb7019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70198660 0x7fcb70198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcb701122e0 con 0x7fcb70102190 2024-04-25T13:15:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70112160 con 0x7fcb70107260 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb76320640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70112460 con 0x7fcb70198660 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 0x7fcb7019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 0x7fcb7019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56602/0 (socket says 172.21.15.38:56602) 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.277+0000 7fcb6ffff640 1 -- 172.21.15.38:0/1257427734 learned_addr learned my addr 172.21.15.38:0/1257427734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6f7fe640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb70107260 0x7fcb7019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb748bd640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70198660 0x7fcb70198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6ffff640 1 -- 172.21.15.38:0/1257427734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70198660 msgr2=0x7fcb70198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6ffff640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70198660 0x7fcb70198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6ffff640 1 -- 172.21.15.38:0/1257427734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb70107260 msgr2=0x7fcb7019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6ffff640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb70107260 0x7fcb7019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6ffff640 1 -- 172.21.15.38:0/1257427734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb70199260 con 0x7fcb70102190 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb748bd640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70198660 0x7fcb70198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6ffff640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 0x7fcb7019e5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000cd80 con 0x7fcb70102190 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb701a5340 con 0x7fcb70102190 2024-04-25T13:15:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcb60010900 con 0x7fcb70102190 2024-04-25T13:15:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb600188d0 con 0x7fcb70102190 2024-04-25T13:15:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.278+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb701a5740 con 0x7fcb70102190 2024-04-25T13:15:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.279+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb70103660 con 0x7fcb70102190 2024-04-25T13:15:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.280+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb60010ab0 con 0x7fcb70102190 2024-04-25T13:15:30.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.281+0000 7fcb6d7fa640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcb580a3d20 0x7fcb580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.281+0000 7fcb6f7fe640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcb580a3d20 0x7fcb580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.282+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fcb600d0c40 con 0x7fcb70102190 2024-04-25T13:15:30.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.282+0000 7fcb6f7fe640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcb580a3d20 0x7fcb580a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb640314b0 tx=0x7fcb640058d0 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:30.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.286+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb6009d2d0 con 0x7fcb70102190 2024-04-25T13:15:30.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:29 smithi104 ceph-mon[32726]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T13:15:30.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:29 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1194183354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:15:30.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:29 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/4169303696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:15:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.477+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb700691c0 con 0x7fcb70102190 2024-04-25T13:15:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.477+0000 7fcb6d7fa640 1 -- 172.21.15.38:0/1257427734 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fcb6009cbd0 con 0x7fcb70102190 2024-04-25T13:15:30.477 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T13:15:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.479+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcb580a3d20 msgr2=0x7fcb580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.479+0000 7fcb76320640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcb580a3d20 0x7fcb580a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb640314b0 tx=0x7fcb640058d0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.479+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 msgr2=0x7fcb7019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.479+0000 7fcb76320640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 0x7fcb7019e5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 shutdown_connections 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fcb580a3d20 0x7fcb580a61e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb70198660 0x7fcb70198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb70107260 0x7fcb7019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 --2- 172.21.15.38:0/1257427734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb70102190 0x7fcb7019e5a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 >> 172.21.15.38:0/1257427734 conn(0x7fcb700793a0 msgr2=0x7fcb700ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 shutdown_connections 2024-04-25T13:15:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.480+0000 7fcb76320640 1 -- 172.21.15.38:0/1257427734 wait complete. 2024-04-25T13:15:30.490 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:15:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 -- 172.21.15.38:0/2543555986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101a30 msgr2=0x7f8368105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2543555986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101a30 0x7f8368105a70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f835c009900 tx=0x7f835c02a410 comp rx=0 tx=0).stop 2024-04-25T13:15:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 -- 172.21.15.38:0/2543555986 shutdown_connections 2024-04-25T13:15:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2543555986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8368106150 0x7f836810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2543555986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101a30 0x7f8368105a70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2543555986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8368101080 0x7f8368101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 -- 172.21.15.38:0/2543555986 >> 172.21.15.38:0/2543555986 conn(0x7f83680771d0 msgr2=0x7f83680775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:30.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 -- 172.21.15.38:0/2543555986 shutdown_connections 2024-04-25T13:15:30.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 -- 172.21.15.38:0/2543555986 wait complete. 2024-04-25T13:15:30.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 Processor -- start 2024-04-25T13:15:30.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.594+0000 7f836d5a2640 1 -- start start 2024-04-25T13:15:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 0x7f836819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8368106150 0x7f836819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83681986a0 0x7f8368198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f83681111b0 con 0x7f8368101080 2024-04-25T13:15:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111030 con 0x7f8368106150 2024-04-25T13:15:30.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111330 con 0x7f83681986a0 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 0x7f836819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 0x7f836819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56612/0 (socket says 172.21.15.38:56612) 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 -- 172.21.15.38:0/2718938629 learned_addr learned my addr 172.21.15.38:0/2718938629 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f83677fe640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83681986a0 0x7f8368198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f83667fc640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8368106150 0x7f836819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 -- 172.21.15.38:0/2718938629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83681986a0 msgr2=0x7f8368198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83681986a0 0x7f8368198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 -- 172.21.15.38:0/2718938629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8368106150 msgr2=0x7f836819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8368106150 0x7f836819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 -- 172.21.15.38:0/2718938629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681992a0 con 0x7f8368101080 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f83677fe640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83681986a0 0x7f8368198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f8366ffd640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 0x7f836819e5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f835800d950 tx=0x7f835800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358017070 con 0x7f8368101080 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8358009c30 con 0x7f8368101080 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a5380 con 0x7f8368101080 2024-04-25T13:15:30.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835800f6b0 con 0x7f8368101080 2024-04-25T13:15:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.595+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a5780 con 0x7f8368101080 2024-04-25T13:15:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.597+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8368102550 con 0x7f8368101080 2024-04-25T13:15:30.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.598+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83580109c0 con 0x7f8368101080 2024-04-25T13:15:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.598+0000 7f834ffff640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f83500a3d20 0x7f83500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.598+0000 7f83667fc640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f83500a3d20 0x7f83500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.599+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f83580d0510 con 0x7f8368101080 2024-04-25T13:15:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.599+0000 7f83667fc640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f83500a3d20 0x7f83500a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f835c0096c0 tx=0x7f835c000bc0 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.602+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8358009de0 con 0x7f8368101080 2024-04-25T13:15:30.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.801+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8368199bd0 con 0x7f8368101080 2024-04-25T13:15:30.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.801+0000 7f834ffff640 1 -- 172.21.15.38:0/2718938629 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+944 (secure 0 0 0) 0x7f8368199bd0 con 0x7f8368101080 2024-04-25T13:15:30.801 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/448455346 2024-04-26T13:04:48.566935+0000 2024-04-25T13:15:30.801 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1381358988 2024-04-26T13:04:48.566935+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3705953772 2024-04-26T13:04:48.566935+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1333793113 2024-04-26T13:04:48.566935+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1475454542 2024-04-26T13:04:48.566935+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/239401667 2024-04-26T12:55:35.192028+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3157707598 2024-04-26T12:55:35.192028+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3913530483 2024-04-26T12:55:08.110358+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2782801986 2024-04-26T12:55:08.110358+0000 2024-04-25T13:15:30.802 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/2310511598 2024-04-26T12:55:08.110358+0000 2024-04-25T13:15:30.803 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/2310511598 2024-04-26T12:55:08.110358+0000 2024-04-25T13:15:30.803 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1333793113 2024-04-26T13:04:48.566935+0000 2024-04-25T13:15:30.803 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1148567137 2024-04-26T12:55:35.192028+0000 2024-04-25T13:15:30.803 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3957422013 2024-04-26T12:55:08.110358+0000 2024-04-25T13:15:30.803 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3157707598 2024-04-26T12:55:35.192028+0000 2024-04-25T13:15:30.803 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2195408134 2024-04-26T12:55:35.192028+0000 2024-04-25T13:15:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:listed 16 entries 2024-04-25T13:15:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f83500a3d20 msgr2=0x7f83500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f83500a3d20 0x7f83500a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f835c0096c0 tx=0x7f835c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 msgr2=0x7f836819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 0x7f836819e5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f835800d950 tx=0x7f835800de10 comp rx=0 tx=0).stop 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 shutdown_connections 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f83500a3d20 0x7f83500a61e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83681986a0 0x7f8368198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8368106150 0x7f836819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 --2- 172.21.15.38:0/2718938629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8368101080 0x7f836819e5e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 >> 172.21.15.38:0/2718938629 conn(0x7f83680771d0 msgr2=0x7f836806a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 shutdown_connections 2024-04-25T13:15:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:30.803+0000 7f836d5a2640 1 -- 172.21.15.38:0/2718938629 wait complete. 2024-04-25T13:15:30.814 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T13:15:30.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:30.814 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T13:15:30.843 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:15:30.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:30.843 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:15:30.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:30.910 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T13:15:30.965 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:15:30.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:30.966 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:15:31.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:31.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-04-25T13:15:31.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:30 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/693346205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:15:31.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:30 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1257427734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:15:31.072 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:30 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2718938629' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:15:31.123 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:15:31.123 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T13:15:31.123 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:15:31.140 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2376 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/e83bf9f91185b190585ca995d0b402c2e84a3bea2ffd7d9ca66b027ca1713749/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/e83bf9f91185b190585ca995d0b402c2e84a3bea2ffd7d9ca66b027ca1713749/diff,workdir=/var/lib/containers/storage/overlay/e83bf9f91185b190585ca995d0b402c2e84a3bea2ffd7d9ca66b027ca1713749/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ca4be1217945c0baad0217ca076b6b675f031eb118baf5db879ed88fdf7f0177/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/ca4be1217945c0baad0217ca076b6b675f031eb118baf5db879ed88fdf7f0177/diff,workdir=/var/lib/containers/storage/overlay/ca4be1217945c0baad0217ca076b6b675f031eb118baf5db879ed88fdf7f0177/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/e51995f67f0db03ad2473a7a945116eccbf795be11124022818f5c39b141c774/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/e51995f67f0db03ad2473a7a945116eccbf795be11124022818f5c39b141c774/diff,workdir=/var/lib/containers/storage/overlay/e51995f67f0db03ad2473a7a945116eccbf795be11124022818f5c39b141c774/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/40daf57d36a891a1db8031d55d7a726683917654a34023a029dd6a86cb7b25eb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/40daf57d36a891a1db8031d55d7a726683917654a34023a029dd6a86cb7b25eb/diff,workdir=/var/lib/containers/storage/overlay/40daf57d36a891a1db8031d55d7a726683917654a34023a029dd6a86cb7b25eb/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5cde7caf8bc26312b5f0d3d984b375f8012b4a77fff6faed07ff28187d84ef7d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/5cde7caf8bc26312b5f0d3d984b375f8012b4a77fff6faed07ff28187d84ef7d/diff,workdir=/var/lib/containers/storage/overlay/5cde7caf8bc26312b5f0d3d984b375f8012b4a77fff6faed07ff28187d84ef7d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4f391c3ffaaf8d98b504ed6bbaef9d8e73c600b73545b431c6cddcbba5085a99/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/4f391c3ffaaf8d98b504ed6bbaef9d8e73c600b73545b431c6cddcbba5085a99/diff,workdir=/var/lib/containers/storage/overlay/4f391c3ffaaf8d98b504ed6bbaef9d8e73c600b73545b431c6cddcbba5085a99/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c0649e4a3abbdf7966cc8914180a4ed01f7fa0fba8db4b0a165933626f4c4f50/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/c0649e4a3abbdf7966cc8914180a4ed01f7fa0fba8db4b0a165933626f4c4f50/diff,workdir=/var/lib/containers/storage/overlay/c0649e4a3abbdf7966cc8914180a4ed01f7fa0fba8db4b0a165933626f4c4f50/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/725db7c77c593d03bfbb119a6f13e16b78ec1c167f9aa95bf99c37c6eb5904fe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/725db7c77c593d03bfbb119a6f13e16b78ec1c167f9aa95bf99c37c6eb5904fe/diff,workdir=/var/lib/containers/storage/overlay/725db7c77c593d03bfbb119a6f13e16b78ec1c167f9aa95bf99c37c6eb5904fe/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f049ba1e766653db351e8461cda26e55b809c2cdfb12c64abaae48ad27996360/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/f049ba1e766653db351e8461cda26e55b809c2cdfb12c64abaae48ad27996360/diff,workdir=/var/lib/containers/storage/overlay/f049ba1e766653db351e8461cda26e55b809c2cdfb12c64abaae48ad27996360/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/92ad4c11e54f1e99fbd585bc82b5f6e99220c9f86229dfc290ceda519575efe6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J6P7HTQVUKT5EBXPKACXFJ2DD3:/var/lib/containers/storage/overlay/l/FEXAVGWBTJKZVJZFM7REJ7WP4Z,upperdir=/var/lib/containers/storage/overlay/92ad4c11e54f1e99fbd585bc82b5f6e99220c9f86229dfc290ceda519575efe6/diff,workdir=/var/lib/containers/storage/overlay/92ad4c11e54f1e99fbd585bc82b5f6e99220c9f86229dfc290ceda519575efe6/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T13:15:31.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:31.140 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T13:15:31.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:15:31.195 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-04-25T13:15:31.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:30 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/693346205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:15:31.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:30 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1257427734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:15:31.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:30 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2718938629' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:15:31.263 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-04-25T13:15:31.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:15:31.264 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T13:15:31.275 INFO:tasks.cephadm:Teardown begin 2024-04-25T13:15:31.275 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:15:31.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:15:31.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:30 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/693346205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:15:31.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:30 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1257427734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:15:31.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:30 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2718938629' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:15:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 -- 172.21.15.38:0/1878498250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324106190 msgr2=0x7f232410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 --2- 172.21.15.38:0/1878498250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324106190 0x7f232410dd60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ca0 tx=0x7f231c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 -- 172.21.15.38:0/1878498250 shutdown_connections 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 --2- 172.21.15.38:0/1878498250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324106190 0x7f232410dd60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 --2- 172.21.15.38:0/1878498250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2324101ac0 0x7f2324105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 --2- 172.21.15.38:0/1878498250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2324101110 0x7f23241014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 -- 172.21.15.38:0/1878498250 >> 172.21.15.38:0/1878498250 conn(0x7f2324077fb0 msgr2=0x7f23240783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 -- 172.21.15.38:0/1878498250 shutdown_connections 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.407+0000 7f232a519640 1 -- 172.21.15.38:0/1878498250 wait complete. 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 Processor -- start 2024-04-25T13:15:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 -- start start 2024-04-25T13:15:31.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 0x7f232419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2324106190 0x7f232419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23241985c0 0x7f2324198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2324111230 con 0x7f2324101110 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23241110b0 con 0x7f2324106190 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f232a519640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23241113b0 con 0x7f23241985c0 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 0x7f232419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 0x7f232419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56638/0 (socket says 172.21.15.38:56638) 2024-04-25T13:15:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 -- 172.21.15.38:0/1885496781 learned_addr learned my addr 172.21.15.38:0/1885496781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f23237fe640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2324106190 0x7f232419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2328ab6640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23241985c0 0x7f2324198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 -- 172.21.15.38:0/1885496781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23241985c0 msgr2=0x7f2324198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23241985c0 0x7f2324198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 -- 172.21.15.38:0/1885496781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2324106190 msgr2=0x7f232419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2324106190 0x7f232419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.408+0000 7f2323fff640 1 -- 172.21.15.38:0/1885496781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23241990e0 con 0x7f2324101110 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.409+0000 7f2323fff640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 0x7f232419e410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f231400c970 tx=0x7f231400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.409+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314007bb0 con 0x7f2324101110 2024-04-25T13:15:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.409+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2314007d50 con 0x7f2324101110 2024-04-25T13:15:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.409+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23241993d0 con 0x7f2324101110 2024-04-25T13:15:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.409+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231400f450 con 0x7f2324101110 2024-04-25T13:15:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.409+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23241a55d0 con 0x7f2324101110 2024-04-25T13:15:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.410+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23240684e0 con 0x7f2324101110 2024-04-25T13:15:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.411+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23140046c0 con 0x7f2324101110 2024-04-25T13:15:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.411+0000 7f23217fa640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f230c0a3d20 0x7f230c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.412+0000 7f23237fe640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f230c0a3d20 0x7f230c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.412+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f23140d0680 con 0x7f2324101110 2024-04-25T13:15:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.412+0000 7f23237fe640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f230c0a3d20 0x7f230c0a61e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2318005d90 tx=0x7f2318005d20 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.416+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2314014030 con 0x7f2324101110 2024-04-25T13:15:31.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.642+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2324101ca0 con 0x7f2324101110 2024-04-25T13:15:31.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.643+0000 7f23217fa640 1 -- 172.21.15.38:0/1885496781 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7f231409cd10 con 0x7f2324101110 2024-04-25T13:15:31.643 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:15:31.643 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:04:47.233398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[14625,24466,24482,24484,24518]},"id":1}]} 2024-04-25T13:15:31.644 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-04-25T13:15:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f230c0a3d20 msgr2=0x7f230c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f230c0a3d20 0x7f230c0a61e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2318005d90 tx=0x7f2318005d20 comp rx=0 tx=0).stop 2024-04-25T13:15:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 msgr2=0x7f232419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 0x7f232419e410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f231400c970 tx=0x7f231400ce30 comp rx=0 tx=0).stop 2024-04-25T13:15:31.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 shutdown_connections 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f230c0a3d20 0x7f230c0a61e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23241985c0 0x7f2324198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2324106190 0x7f232419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 --2- 172.21.15.38:0/1885496781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2324101110 0x7f232419e410 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 >> 172.21.15.38:0/1885496781 conn(0x7f2324077fb0 msgr2=0x7f2324104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 shutdown_connections 2024-04-25T13:15:31.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.645+0000 7f232a519640 1 -- 172.21.15.38:0/1885496781 wait complete. 2024-04-25T13:15:31.656 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:15:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 -- 172.21.15.38:0/1238091388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 msgr2=0x7fc5dc10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/1238091388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc10bfe0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009900 tx=0x7fc5d002a510 comp rx=0 tx=0).stop 2024-04-25T13:15:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 -- 172.21.15.38:0/1238091388 shutdown_connections 2024-04-25T13:15:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/1238091388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5dc10c520 0x7fc5dc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/1238091388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc10bfe0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/1238091388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5dc1031e0 0x7fc5dc1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 -- 172.21.15.38:0/1238091388 >> 172.21.15.38:0/1238091388 conn(0x7fc5dc077f50 msgr2=0x7fc5dc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 -- 172.21.15.38:0/1238091388 shutdown_connections 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.758+0000 7fc5e08e1640 1 -- 172.21.15.38:0/1238091388 wait complete. 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 Processor -- start 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 -- start start 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5dc1031e0 0x7fc5dc19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5dc10c520 0x7fc5dc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1122f0 con 0x7fc5dc103b00 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc112170 con 0x7fc5dc10c520 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5e08e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc112470 con 0x7fc5dc1031e0 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5d9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5d9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56658/0 (socket says 172.21.15.38:56658) 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.759+0000 7fc5d9d9b640 1 -- 172.21.15.38:0/3442424796 learned_addr learned my addr 172.21.15.38:0/3442424796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5da59c640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5dc1031e0 0x7fc5dc19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5dad9d640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5dc10c520 0x7fc5dc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5d9d9b640 1 -- 172.21.15.38:0/3442424796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5dc1031e0 msgr2=0x7fc5dc19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5d9d9b640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5dc1031e0 0x7fc5dc19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5d9d9b640 1 -- 172.21.15.38:0/3442424796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5dc10c520 msgr2=0x7fc5dc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5d9d9b640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5dc10c520 0x7fc5dc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5d9d9b640 1 -- 172.21.15.38:0/3442424796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc198f10 con 0x7fc5dc103b00 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5d9d9b640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc19ead0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0005ab0 tx=0x7fc5d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d003f030 con 0x7fc5dc103b00 2024-04-25T13:15:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc5d0003cd0 con 0x7fc5dc103b00 2024-04-25T13:15:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc1991a0 con 0x7fc5dc103b00 2024-04-25T13:15:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0035070 con 0x7fc5dc103b00 2024-04-25T13:15:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.760+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1a5330 con 0x7fc5dc103b00 2024-04-25T13:15:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.761+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0698e0 con 0x7fc5dc103b00 2024-04-25T13:15:31.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.762+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5d0042030 con 0x7fc5dc103b00 2024-04-25T13:15:31.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.763+0000 7fc5cb7fe640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc5c40a3d20 0x7fc5c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:31.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.763+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc5d00f0d00 con 0x7fc5dc103b00 2024-04-25T13:15:31.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.763+0000 7fc5da59c640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc5c40a3d20 0x7fc5c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:31.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.763+0000 7fc5da59c640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc5c40a3d20 0x7fc5c40a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00bbe0 tx=0x7fc5cc00b390 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:31.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.768+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5d00bd390 con 0x7fc5dc103b00 2024-04-25T13:15:31.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.998+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5dc02d5c0 con 0x7fc5dc103b00 2024-04-25T13:15:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:31.998+0000 7fc5cb7fe640 1 -- 172.21.15.38:0/3442424796 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7fc5d00bcc90 con 0x7fc5dc103b00 2024-04-25T13:15:31.998 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:15:31.999 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3785884353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3785884353},{"type":"v1","addr":"172.21.15.38:6835","nonce":3785884353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/1490443710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":1490443710},{"type":"v1","addr":"172.21.15.38:6837","nonce":1490443710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:02:44.261324+0000","modified":"2024-04-25T13:04:47.233398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24484,"mds_2":24482,"mds_3":24466,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/3427364907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3427364907},{"type":"v1","addr":"172.21.15.38:6839","nonce":3427364907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.38:6841/4187315479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":4187315479},{"type":"v1","addr":"172.21.15.38:6841","nonce":4187315479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/2494647672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2494647672},{"type":"v1","addr":"172.21.15.178:6833","nonce":2494647672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/650369302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":650369302},{"type":"v1","addr":"172.21.15.104:6833","nonce":650369302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/4172911078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":4172911078},{"type":"v1","addr":"172.21.15.104:6835","nonce":4172911078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2362992725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2362992725},{"type":"v1","addr":"172.21.15.178:6835","nonce":2362992725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6837/2053792208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2053792208},{"type":"v1","addr":"172.21.15.178:6837","nonce":2053792208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6837/2247876378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2247876378},{"type":"v1","addr":"172.21.15.104:6837","nonce":2247876378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/391098246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":391098246},{"type":"v1","addr":"172.21.15.178:6839","nonce":391098246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1961595450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1961595450},{"type":"v1","addr":"172.21.15.104:6839","nonce":1961595450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[14625,24466,24482,24484,24518]},"id":1}]} 2024-04-25T13:15:31.999 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc5c40a3d20 msgr2=0x7fc5c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc5c40a3d20 0x7fc5c40a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00bbe0 tx=0x7fc5cc00b390 comp rx=0 tx=0).stop 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 msgr2=0x7fc5dc19ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc19ead0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0005ab0 tx=0x7fc5d00042f0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 shutdown_connections 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5dc10c520 0x7fc5dc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc103b00 0x7fc5dc19ead0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fc5c40a3d20 0x7fc5c40a61e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 --2- 172.21.15.38:0/3442424796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5dc1031e0 0x7fc5dc19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 >> 172.21.15.38:0/3442424796 conn(0x7fc5dc077f50 msgr2=0x7fc5dc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:32.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 shutdown_connections 2024-04-25T13:15:32.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.001+0000 7fc5e08e1640 1 -- 172.21.15.38:0/3442424796 wait complete. 2024-04-25T13:15:32.012 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:15:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 -- 172.21.15.38:0/318969970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430101a50 msgr2=0x7f9430105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 --2- 172.21.15.38:0/318969970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430101a50 0x7f9430105a90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9424009900 tx=0x7f942402a410 comp rx=0 tx=0).stop 2024-04-25T13:15:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 -- 172.21.15.38:0/318969970 shutdown_connections 2024-04-25T13:15:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 --2- 172.21.15.38:0/318969970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9430106170 0x7f943010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 --2- 172.21.15.38:0/318969970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430101a50 0x7f9430105a90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 --2- 172.21.15.38:0/318969970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94301010a0 0x7f9430101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 -- 172.21.15.38:0/318969970 >> 172.21.15.38:0/318969970 conn(0x7f9430077f50 msgr2=0x7f9430078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 -- 172.21.15.38:0/318969970 shutdown_connections 2024-04-25T13:15:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.114+0000 7f9434c60640 1 -- 172.21.15.38:0/318969970 wait complete. 2024-04-25T13:15:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 Processor -- start 2024-04-25T13:15:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 -- start start 2024-04-25T13:15:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94301010a0 0x7f943019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 0x7f943019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9430198690 0x7f9430198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9430111270 con 0x7f9430106170 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94301110f0 con 0x7f9430198690 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f9434c60640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94301113f0 con 0x7f94301010a0 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 0x7f943019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 0x7f943019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56680/0 (socket says 172.21.15.38:56680) 2024-04-25T13:15:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 -- 172.21.15.38:0/2580363222 learned_addr learned my addr 172.21.15.38:0/2580363222 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942ed9d640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9430198690 0x7f9430198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942e59c640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94301010a0 0x7f943019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 -- 172.21.15.38:0/2580363222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94301010a0 msgr2=0x7f943019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94301010a0 0x7f943019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 -- 172.21.15.38:0/2580363222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9430198690 msgr2=0x7f9430198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9430198690 0x7f9430198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 -- 172.21.15.38:0/2580363222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9430199290 con 0x7f9430106170 2024-04-25T13:15:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.115+0000 7f942dd9b640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 0x7f943019eb10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9424005940 tx=0x7f9424004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.116+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942403f030 con 0x7f9430106170 2024-04-25T13:15:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.116+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9430199520 con 0x7f9430106170 2024-04-25T13:15:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.116+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9424003cd0 con 0x7f9430106170 2024-04-25T13:15:32.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.116+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424035070 con 0x7f9430106170 2024-04-25T13:15:32.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.116+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301a5660 con 0x7f9430106170 2024-04-25T13:15:32.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.117+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9430102570 con 0x7f9430106170 2024-04-25T13:15:32.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.118+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9424042030 con 0x7f9430106170 2024-04-25T13:15:32.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.118+0000 7f941f7fe640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f94180a3d20 0x7f94180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.118+0000 7f942e59c640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f94180a3d20 0x7f94180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.118+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f94240f1110 con 0x7f9430106170 2024-04-25T13:15:32.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.119+0000 7f942e59c640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f94180a3d20 0x7f94180a61e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9420009990 tx=0x7f9420009110 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:32.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.123+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94240bd840 con 0x7f9430106170 2024-04-25T13:15:32.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:31 smithi178 ceph-mon[33233]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:15:32.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:31 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/1885496781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:15:32.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.311+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9430101a50 con 0x7f9430106170 2024-04-25T13:15:32.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.312+0000 7f941f7fe640 1 -- 172.21.15.38:0/2580363222 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23117 (secure 0 0 0) 0x7f94240bd140 con 0x7f9430106170 2024-04-25T13:15:32.313 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T13:15:32.313 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":99,"fsid":"d3627c2a-0302-11ef-bc93-c7b262605968","created":"2024-04-25T12:54:40.451583+0000","modified":"2024-04-25T13:15:15.878896+0000","last_up_change":"2024-04-25T13:01:55.654712+0000","last_in_change":"2024-04-25T13:01:36.186093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:58:01.286924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:02:42.235731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:02:43.206179+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc0ea8fe-71f4-4976-8844-d906980de942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6803","nonce":4061500171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6805","nonce":4061500171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6809","nonce":4061500171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4061500171},{"type":"v1","addr":"172.21.15.38:6807","nonce":4061500171}]},"public_addr":"172.21.15.38:6803/4061500171","cluster_addr":"172.21.15.38:6805/4061500171","heartbeat_back_addr":"172.21.15.38:6809/4061500171","heartbeat_front_addr":"172.21.15.38:6807/4061500171","state":["exists","up"]},{"osd":1,"uuid":"61eee122-04d2-413a-9203-29184e87e08f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6801","nonce":739328309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6803","nonce":739328309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6807","nonce":739328309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":739328309},{"type":"v1","addr":"172.21.15.104:6805","nonce":739328309}]},"public_addr":"172.21.15.104:6801/739328309","cluster_addr":"172.21.15.104:6803/739328309","heartbeat_back_addr":"172.21.15.104:6807/739328309","heartbeat_front_addr":"172.21.15.104:6805/739328309","state":["exists","up"]},{"osd":2,"uuid":"d0d8bc3e-b869-46b9-b674-3f547e26ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6801","nonce":760031576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6803","nonce":760031576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6807","nonce":760031576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":760031576},{"type":"v1","addr":"172.21.15.178:6805","nonce":760031576}]},"public_addr":"172.21.15.178:6801/760031576","cluster_addr":"172.21.15.178:6803/760031576","heartbeat_back_addr":"172.21.15.178:6807/760031576","heartbeat_front_addr":"172.21.15.178:6805/760031576","state":["exists","up"]},{"osd":3,"uuid":"408f9b8b-64f3-4daa-b1ec-6c4860a8fe4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943053505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943053505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6817","nonce":1943053505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943053505},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943053505}]},"public_addr":"172.21.15.38:6811/1943053505","cluster_addr":"172.21.15.38:6813/1943053505","heartbeat_back_addr":"172.21.15.38:6817/1943053505","heartbeat_front_addr":"172.21.15.38:6815/1943053505","state":["exists","up"]},{"osd":4,"uuid":"e42ba42a-a706-42ba-84d0-85ef48660c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6809","nonce":309827173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6811","nonce":309827173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6815","nonce":309827173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":309827173},{"type":"v1","addr":"172.21.15.104:6813","nonce":309827173}]},"public_addr":"172.21.15.104:6809/309827173","cluster_addr":"172.21.15.104:6811/309827173","heartbeat_back_addr":"172.21.15.104:6815/309827173","heartbeat_front_addr":"172.21.15.104:6813/309827173","state":["exists","up"]},{"osd":5,"uuid":"93a59196-6a9a-4233-ad44-dee831b56089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6809","nonce":1085104409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6811","nonce":1085104409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6815","nonce":1085104409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1085104409},{"type":"v1","addr":"172.21.15.178:6813","nonce":1085104409}]},"public_addr":"172.21.15.178:6809/1085104409","cluster_addr":"172.21.15.178:6811/1085104409","heartbeat_back_addr":"172.21.15.178:6815/1085104409","heartbeat_front_addr":"172.21.15.178:6813/1085104409","state":["exists","up"]},{"osd":6,"uuid":"fcf8f123-b8f9-49e5-bca5-34f32285c0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6819","nonce":2531974702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6821","nonce":2531974702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6825","nonce":2531974702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2531974702},{"type":"v1","addr":"172.21.15.38:6823","nonce":2531974702}]},"public_addr":"172.21.15.38:6819/2531974702","cluster_addr":"172.21.15.38:6821/2531974702","heartbeat_back_addr":"172.21.15.38:6825/2531974702","heartbeat_front_addr":"172.21.15.38:6823/2531974702","state":["exists","up"]},{"osd":7,"uuid":"2f074366-1e63-4ad6-9f21-30018b0cb493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6817","nonce":3955477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6819","nonce":3955477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6823","nonce":3955477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":3955477},{"type":"v1","addr":"172.21.15.104:6821","nonce":3955477}]},"public_addr":"172.21.15.104:6817/3955477","cluster_addr":"172.21.15.104:6819/3955477","heartbeat_back_addr":"172.21.15.104:6823/3955477","heartbeat_front_addr":"172.21.15.104:6821/3955477","state":["exists","up"]},{"osd":8,"uuid":"09d79a36-ce86-4e9a-b60a-6a709d437bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6817","nonce":2770931166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6819","nonce":2770931166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6823","nonce":2770931166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2770931166},{"type":"v1","addr":"172.21.15.178:6821","nonce":2770931166}]},"public_addr":"172.21.15.178:6817/2770931166","cluster_addr":"172.21.15.178:6819/2770931166","heartbeat_back_addr":"172.21.15.178:6823/2770931166","heartbeat_front_addr":"172.21.15.178:6821/2770931166","state":["exists","up"]},{"osd":9,"uuid":"47dbcc3a-1a6a-4896-a5d2-9866e080fd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6827","nonce":1397570004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6829","nonce":1397570004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6833","nonce":1397570004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1397570004},{"type":"v1","addr":"172.21.15.38:6831","nonce":1397570004}]},"public_addr":"172.21.15.38:6827/1397570004","cluster_addr":"172.21.15.38:6829/1397570004","heartbeat_back_addr":"172.21.15.38:6833/1397570004","heartbeat_front_addr":"172.21.15.38:6831/1397570004","state":["exists","up"]},{"osd":10,"uuid":"38c90875-61d4-402f-bee4-af2adb22e185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6825","nonce":2081501725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6827","nonce":2081501725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6831","nonce":2081501725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2081501725},{"type":"v1","addr":"172.21.15.104:6829","nonce":2081501725}]},"public_addr":"172.21.15.104:6825/2081501725","cluster_addr":"172.21.15.104:6827/2081501725","heartbeat_back_addr":"172.21.15.104:6831/2081501725","heartbeat_front_addr":"172.21.15.104:6829/2081501725","state":["exists","up"]},{"osd":11,"uuid":"f03efb23-83a5-460c-90be-6aef89f88698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6825","nonce":700919223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6827","nonce":700919223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6831","nonce":700919223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":700919223},{"type":"v1","addr":"172.21.15.178:6829","nonce":700919223}]},"public_addr":"172.21.15.178:6825/700919223","cluster_addr":"172.21.15.178:6827/700919223","heartbeat_back_addr":"172.21.15.178:6831/700919223","heartbeat_front_addr":"172.21.15.178:6829/700919223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:09.846998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:34.002685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:57:58.204631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:24.187628+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:58:49.900765+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:15.180029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:59:41.615455+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:08.160653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:00:34.293459+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:01.972293+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:28.390835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:01:54.390616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/448455346":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1381358988":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/3705953772":"2024-04-26T13:04:48.566935+0000","172.21.15.38:6801/1333793113":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1475454542":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/239401667":"2024-04-26T12:55:35.192028+0000","172.21.15.38:6800/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3913530483":"2024-04-26T12:55:08.110358+0000","172.21.15.38:0/2782801986":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/2310511598":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6800/1333793113":"2024-04-26T13:04:48.566935+0000","172.21.15.38:0/1148567137":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/3957422013":"2024-04-26T12:55:08.110358+0000","172.21.15.38:6801/3157707598":"2024-04-26T12:55:35.192028+0000","172.21.15.38:0/2195408134":"2024-04-26T12:55:35.192028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:15:32.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f94180a3d20 msgr2=0x7f94180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f94180a3d20 0x7f94180a61e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9420009990 tx=0x7f9420009110 comp rx=0 tx=0).stop 2024-04-25T13:15:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 msgr2=0x7f943019eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 0x7f943019eb10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9424005940 tx=0x7f9424004360 comp rx=0 tx=0).stop 2024-04-25T13:15:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 shutdown_connections 2024-04-25T13:15:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9430198690 0x7f9430198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9430106170 0x7f943019eb10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f94180a3d20 0x7f94180a61e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 --2- 172.21.15.38:0/2580363222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94301010a0 0x7f943019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 >> 172.21.15.38:0/2580363222 conn(0x7f9430077f50 msgr2=0x7f9430069440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 shutdown_connections 2024-04-25T13:15:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.314+0000 7f9434c60640 1 -- 172.21.15.38:0/2580363222 wait complete. 2024-04-25T13:15:32.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:31 smithi038 ceph-mon[29400]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:15:32.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:31 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/1885496781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:15:32.326 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:15:32.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:31 smithi104 ceph-mon[32726]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T13:15:32.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:31 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/1885496781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:15:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.430+0000 7fdd34720640 1 -- 172.21.15.38:0/2392820278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c069620 msgr2=0x7fdd2c069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- 172.21.15.38:0/2392820278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c069620 0x7fdd2c069a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009900 tx=0x7fdd1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:15:32.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- 172.21.15.38:0/2392820278 shutdown_connections 2024-04-25T13:15:32.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- 172.21.15.38:0/2392820278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c10e750 0x7fdd2c110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- 172.21.15.38:0/2392820278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c069620 0x7fdd2c069a80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- 172.21.15.38:0/2392820278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c102df0 0x7fdd2c0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- 172.21.15.38:0/2392820278 >> 172.21.15.38:0/2392820278 conn(0x7fdd2c077cb0 msgr2=0x7fdd2c0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- 172.21.15.38:0/2392820278 shutdown_connections 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- 172.21.15.38:0/2392820278 wait complete. 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 Processor -- start 2024-04-25T13:15:32.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- start start 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c069620 0x7fdd2c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c102df0 0x7fdd2c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 0x7fdd2c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c114490 con 0x7fdd2c10e750 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c114310 con 0x7fdd2c069620 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.431+0000 7fdd34720640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c114610 con 0x7fdd2c102df0 2024-04-25T13:15:32.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 0x7fdd2c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 0x7fdd2c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56700/0 (socket says 172.21.15.38:56700) 2024-04-25T13:15:32.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 -- 172.21.15.38:0/3016158169 learned_addr learned my addr 172.21.15.38:0/3016158169 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:32.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd31cbb640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c102df0 0x7fdd2c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd324bc640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c069620 0x7fdd2c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 -- 172.21.15.38:0/3016158169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c102df0 msgr2=0x7fdd2c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c102df0 0x7fdd2c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 -- 172.21.15.38:0/3016158169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c069620 msgr2=0x7fdd2c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c069620 0x7fdd2c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 -- 172.21.15.38:0/3016158169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c19af60 con 0x7fdd2c10e750 2024-04-25T13:15:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd324bc640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c069620 0x7fdd2c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:15:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd32cbd640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 0x7fdd2c19a790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800d750 tx=0x7fdd2800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:32.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28017070 con 0x7fdd2c10e750 2024-04-25T13:15:32.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd280054f0 con 0x7fdd2c10e750 2024-04-25T13:15:32.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c19b250 con 0x7fdd2c10e750 2024-04-25T13:15:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28004cd0 con 0x7fdd2c10e750 2024-04-25T13:15:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.432+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c1a7470 con 0x7fdd2c10e750 2024-04-25T13:15:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.433+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2c101460 con 0x7fdd2c10e750 2024-04-25T13:15:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.434+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd28009bd0 con 0x7fdd2c10e750 2024-04-25T13:15:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.435+0000 7fdd237fe640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdd180a3d20 0x7fdd180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.435+0000 7fdd324bc640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdd180a3d20 0x7fdd180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.435+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fdd280d02c0 con 0x7fdd2c10e750 2024-04-25T13:15:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.436+0000 7fdd324bc640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdd180a3d20 0x7fdd180a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd24008e00 tx=0x7fdd24008350 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.440+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd2809c8f0 con 0x7fdd2c10e750 2024-04-25T13:15:32.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.664+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdd2c1a7640 con 0x7fdd2c10e750 2024-04-25T13:15:32.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.666+0000 7fdd237fe640 1 -- 172.21.15.38:0/3016158169 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fdd2809c1f0 con 0x7fdd2c10e750 2024-04-25T13:15:32.666 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:15:32.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdd180a3d20 msgr2=0x7fdd180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdd180a3d20 0x7fdd180a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd24008e00 tx=0x7fdd24008350 comp rx=0 tx=0).stop 2024-04-25T13:15:32.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 msgr2=0x7fdd2c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 0x7fdd2c19a790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800d750 tx=0x7fdd2800dc10 comp rx=0 tx=0).stop 2024-04-25T13:15:32.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 shutdown_connections 2024-04-25T13:15:32.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd2c10e750 0x7fdd2c19a790 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdd180a3d20 0x7fdd180a61e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c102df0 0x7fdd2c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 --2- 172.21.15.38:0/3016158169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd2c069620 0x7fdd2c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 >> 172.21.15.38:0/3016158169 conn(0x7fdd2c077cb0 msgr2=0x7fdd2c10f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:32.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 shutdown_connections 2024-04-25T13:15:32.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.669+0000 7fdd34720640 1 -- 172.21.15.38:0/3016158169 wait complete. 2024-04-25T13:15:32.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:15:32.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 -- 172.21.15.38:0/2636669680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc102bc0 msgr2=0x7fdbbc106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2636669680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc102bc0 0x7fdbbc106c00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009900 tx=0x7fdbb002a410 comp rx=0 tx=0).stop 2024-04-25T13:15:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 -- 172.21.15.38:0/2636669680 shutdown_connections 2024-04-25T13:15:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2636669680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbbc1072e0 0x7fdbbc10eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2636669680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc102bc0 0x7fdbbc106c00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2636669680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 -- 172.21.15.38:0/2636669680 >> 172.21.15.38:0/2636669680 conn(0x7fdbbc0fdab0 msgr2=0x7fdbbc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 -- 172.21.15.38:0/2636669680 shutdown_connections 2024-04-25T13:15:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 -- 172.21.15.38:0/2636669680 wait complete. 2024-04-25T13:15:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 Processor -- start 2024-04-25T13:15:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.796+0000 7fdbc1860640 1 -- start start 2024-04-25T13:15:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc1860640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc1860640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbbc1072e0 0x7fdbbc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc1860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc198740 0x7fdbbc198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc1860640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc112360 con 0x7fdbbc198740 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc1860640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1121e0 con 0x7fdbbc1072e0 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc1860640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1124e0 con 0x7fdbbc102210 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc198740 0x7fdbbc198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.38:42124/0 (socket says 172.21.15.38:42124) 2024-04-25T13:15:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 -- 172.21.15.38:0/2686414645 learned_addr learned my addr 172.21.15.38:0/2686414645 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbba7fc640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbbc1072e0 0x7fdbbc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 -- 172.21.15.38:0/2686414645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbbc1072e0 msgr2=0x7fdbbc19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbbc1072e0 0x7fdbbc19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 -- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc198740 msgr2=0x7fdbbc198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc198740 0x7fdbbc198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 -- 172.21.15.38:0/2686414645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc199340 con 0x7fdbbc102210 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbb7fe640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc198740 0x7fdbbc198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:15:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbbaffd640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc19e680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00e9f0 tx=0x7fdbac00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.797+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00cd80 con 0x7fdbbc102210 2024-04-25T13:15:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.798+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc1a5420 con 0x7fdbbc102210 2024-04-25T13:15:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.798+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdbac0054f0 con 0x7fdbbc102210 2024-04-25T13:15:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.798+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac010690 con 0x7fdbbc102210 2024-04-25T13:15:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.798+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a57d0 con 0x7fdbbc102210 2024-04-25T13:15:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.799+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc1036e0 con 0x7fdbbc102210 2024-04-25T13:15:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.800+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdbac0046c0 con 0x7fdbbc102210 2024-04-25T13:15:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.800+0000 7fdbc085e640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdba40a3d20 0x7fdba40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.800+0000 7fdbba7fc640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdba40a3d20 0x7fdba40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.800+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8928+0+0 (secure 0 0 0) 0x7fdbac0d0cf0 con 0x7fdbbc102210 2024-04-25T13:15:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.801+0000 7fdbba7fc640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdba40a3d20 0x7fdba40a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00314b0 tx=0x7fdbb00058d0 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:32.804+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbac09d1c0 con 0x7fdbbc102210 2024-04-25T13:15:32.933 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a[29396]: 2024-04-25T13:15:32.664+0000 7f410c84b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:15:32.934 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g[68104]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:32.934 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g[68104]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:32.934 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mds[68108]: -- 172.21.15.38:0/2584874549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cab1a83a40 con 0x55cab1a56c00 2024-04-25T13:15:32.935 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g[68104]: starting mds.g at 2024-04-25T13:15:32.935 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a[68455]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:32.935 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a[68455]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:32.935 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mds[68459]: -- 172.21.15.38:0/1340252889 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564984dbba40 con 0x564984d8e800 2024-04-25T13:15:32.935 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a[68455]: starting mds.a at 2024-04-25T13:15:32.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3442424796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:15:32.990 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2580363222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:15:32.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3016158169' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:15:32.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:15:32.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:15:32.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/3016158169' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:15:32.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:15:32.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mon[33233]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T13:15:32.992 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f[63694]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:32.992 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f[63694]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:32.992 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mds[63698]: -- 172.21.15.178:0/1433940160 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5586438804e0 con 0x558643857000 2024-04-25T13:15:32.992 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mds[63698]: -- 172.21.15.178:0/1433940160 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558643883a40 con 0x558643857000 2024-04-25T13:15:32.992 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f[63694]: starting mds.f at 2024-04-25T13:15:32.992 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c[64007]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:32.993 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c[64007]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:32.993 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mds[64011]: -- 172.21.15.178:0/1619287932 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563022fda40 con 0x5563022d0c00 2024-04-25T13:15:32.993 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c[64007]: starting mds.c at 2024-04-25T13:15:32.993 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i[64321]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:32.993 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i[64321]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:32.993 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mds[64325]: -- 172.21.15.178:0/1692820634 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563896623a40 con 0x5638965f6800 2024-04-25T13:15:32.994 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i[64321]: starting mds.i at 2024-04-25T13:15:32.994 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l[63372]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:32.994 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l[63372]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:32.994 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-mds[63376]: -- 172.21.15.178:0/3996137124 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d93ca0ba40 con 0x55d93c9de800 2024-04-25T13:15:32.994 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:15:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l[63372]: starting mds.l at 2024-04-25T13:15:33.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.030+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdbbc068d70 con 0x7fdbbc102210 2024-04-25T13:15:33.075 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k[62322]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:33.075 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k[62322]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:33.075 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mds[62326]: -- 172.21.15.104:0/2973178135 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f15e01ba40 con 0x55f15dfee800 2024-04-25T13:15:33.075 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k[62322]: starting mds.k at 2024-04-25T13:15:33.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3442424796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:15:33.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2580363222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:15:33.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3016158169' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:15:33.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:15:33.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:15:33.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/3016158169' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:15:33.077 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:15:33.077 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mon[32726]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T13:15:33.077 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h[62957]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:33.077 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h[62957]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:33.077 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mds[62961]: -- 172.21.15.104:0/2827342833 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567c7227a40 con 0x5567c71fb000 2024-04-25T13:15:33.077 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h[62957]: starting mds.h at 2024-04-25T13:15:33.078 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:33.078 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:33.078 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mds[63273]: -- 172.21.15.104:0/3687130881 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8d35e7a40 con 0x55d8d35ba800 2024-04-25T13:15:33.078 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: starting mds.b at 2024-04-25T13:15:33.079 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e[62650]: ignoring --setuser ceph since I am not root 2024-04-25T13:15:33.079 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e[62650]: ignoring --setgroup ceph since I am not root 2024-04-25T13:15:33.079 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-mds[62654]: -- 172.21.15.104:0/26213633 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563092ecfa40 con 0x563092ea2800 2024-04-25T13:15:33.079 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:15:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e[62650]: starting mds.e at 2024-04-25T13:15:33.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3442424796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2580363222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3016158169' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/3016158169' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:15:33.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:32 smithi038 ceph-mon[29400]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T13:15:33.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.675+0000 7fdbc085e640 1 -- 172.21.15.38:0/2686414645 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fdbac09cac0 con 0x7fdbbc102210 2024-04-25T13:15:33.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdba40a3d20 msgr2=0x7fdba40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:33.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdba40a3d20 0x7fdba40a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00314b0 tx=0x7fdbb00058d0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 msgr2=0x7fdbbc19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc19e680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00e9f0 tx=0x7fdbac00edf0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 shutdown_connections 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbbc198740 0x7fdbbc198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbbc1072e0 0x7fdbbc19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7fdba40a3d20 0x7fdba40a61e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 --2- 172.21.15.38:0/2686414645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102210 0x7fdbbc19e680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.678+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 >> 172.21.15.38:0/2686414645 conn(0x7fdbbc0fdab0 msgr2=0x7fdbbc0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.679+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 shutdown_connections 2024-04-25T13:15:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.679+0000 7fdbc1860640 1 -- 172.21.15.38:0/2686414645 wait complete. 2024-04-25T13:15:33.695 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:15:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 -- 172.21.15.38:0/478941712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c101090 msgr2=0x7f812c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 --2- 172.21.15.38:0/478941712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c101090 0x7f812c101470 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8114009900 tx=0x7f811402a3f0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 -- 172.21.15.38:0/478941712 shutdown_connections 2024-04-25T13:15:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 --2- 172.21.15.38:0/478941712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c106160 0x7f812c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 --2- 172.21.15.38:0/478941712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f812c101a40 0x7f812c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 --2- 172.21.15.38:0/478941712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c101090 0x7f812c101470 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 -- 172.21.15.38:0/478941712 >> 172.21.15.38:0/478941712 conn(0x7f812c077f50 msgr2=0x7f812c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 -- 172.21.15.38:0/478941712 shutdown_connections 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.809+0000 7f8132fef640 1 -- 172.21.15.38:0/478941712 wait complete. 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 Processor -- start 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 -- start start 2024-04-25T13:15:33.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f812c101090 0x7f812c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 0x7f812c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c1035a0 0x7f812c1a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f812c111200 con 0x7f812c106160 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f812c111080 con 0x7f812c1035a0 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8132fef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f812c111380 con 0x7f812c101090 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 0x7f812c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:33.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 0x7f812c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56770/0 (socket says 172.21.15.38:56770) 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 -- 172.21.15.38:0/160564945 learned_addr learned my addr 172.21.15.38:0/160564945 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f8130d8b640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f812c101090 0x7f812c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f813158c640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c1035a0 0x7f812c1a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 -- 172.21.15.38:0/160564945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f812c101090 msgr2=0x7f812c102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f812c101090 0x7f812c102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 -- 172.21.15.38:0/160564945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c1035a0 msgr2=0x7f812c1a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c1035a0 0x7f812c1a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f812bfff640 1 -- 172.21.15.38:0/160564945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c1a70e0 con 0x7f812c106160 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.810+0000 7f813158c640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c1035a0 0x7f812c1a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.811+0000 7f812bfff640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 0x7f812c103060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f811c00d740 tx=0x7f811c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.811+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c017070 con 0x7f812c106160 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.811+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f811c00de50 con 0x7f812c106160 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.811+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c1a73d0 con 0x7f812c106160 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.811+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c004d90 con 0x7f812c106160 2024-04-25T13:15:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.811+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c1a78d0 con 0x7f812c106160 2024-04-25T13:15:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.812+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812c068d70 con 0x7f812c106160 2024-04-25T13:15:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.813+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f811c004f30 con 0x7f812c106160 2024-04-25T13:15:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.814+0000 7f8129ffb640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f81180a3d50 0x7f81180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.814+0000 7f8130d8b640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f81180a3d50 0x7f81180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.814+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8928+0+0 (secure 0 0 0) 0x7f811c0d02d0 con 0x7f812c106160 2024-04-25T13:15:33.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.815+0000 7f8130d8b640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f81180a3d50 0x7f81180a6210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8114009230 tx=0x7f81140058d0 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:33.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:33.819+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f811c09c820 con 0x7f812c106160 2024-04-25T13:15:33.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:15:33.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/2686414645' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:15:33.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:15:33.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:15:33.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:15:33.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: Cluster is now healthy 2024-04-25T13:15:33.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:15:33.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6839/987017597,v1:172.21.15.38:6840/987017597] up:boot 2024-04-25T13:15:33.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.38:6838/2499821808,v1:172.21.15.38:6841/2499821808] up:boot 2024-04-25T13:15:33.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6833/656750253,v1:172.21.15.104:6835/656750253] up:boot 2024-04-25T13:15:33.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6832/2089768912,v1:172.21.15.104:6834/2089768912] up:boot 2024-04-25T13:15:33.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6832/2306780582,v1:172.21.15.178:6834/2306780582] up:boot 2024-04-25T13:15:33.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6833/2204870830,v1:172.21.15.178:6835/2204870830] up:boot 2024-04-25T13:15:33.987 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6836/199999798,v1:172.21.15.178:6837/199999798] up:boot 2024-04-25T13:15:33.987 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.178:6838/3804315653,v1:172.21.15.178:6839/3804315653] up:boot 2024-04-25T13:15:33.987 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6836/99741491,v1:172.21.15.104:6838/99741491] up:boot 2024-04-25T13:15:33.987 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: mds.? [v2:172.21.15.104:6837/3835401869,v1:172.21.15.104:6839/3835401869] up:boot 2024-04-25T13:15:33.987 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: fsmap 12 up:standby 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:15:33.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:15:33.989 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:15:33.989 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:15:33.989 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:33 smithi038 ceph-mon[29400]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:15:34.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.006+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f812c10a3a0 con 0x7f812c106160 2024-04-25T13:15:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:15:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/2686414645' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:15:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:15:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:15:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:15:34.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: Cluster is now healthy 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6839/987017597,v1:172.21.15.38:6840/987017597] up:boot 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.38:6838/2499821808,v1:172.21.15.38:6841/2499821808] up:boot 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6833/656750253,v1:172.21.15.104:6835/656750253] up:boot 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6832/2089768912,v1:172.21.15.104:6834/2089768912] up:boot 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6832/2306780582,v1:172.21.15.178:6834/2306780582] up:boot 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6833/2204870830,v1:172.21.15.178:6835/2204870830] up:boot 2024-04-25T13:15:34.242 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6836/199999798,v1:172.21.15.178:6837/199999798] up:boot 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.178:6838/3804315653,v1:172.21.15.178:6839/3804315653] up:boot 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6836/99741491,v1:172.21.15.104:6838/99741491] up:boot 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: mds.? [v2:172.21.15.104:6837/3835401869,v1:172.21.15.104:6839/3835401869] up:boot 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: fsmap 12 up:standby 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:15:34.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:15:34.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:15:34.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:15:34.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:15:34.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:15:34.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:15:34.244 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:33 smithi178 ceph-mon[33233]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:15:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:15:34.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/2686414645' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: Cluster is now healthy 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6839/987017597,v1:172.21.15.38:6840/987017597] up:boot 2024-04-25T13:15:34.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.38:6838/2499821808,v1:172.21.15.38:6841/2499821808] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6833/656750253,v1:172.21.15.104:6835/656750253] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6832/2089768912,v1:172.21.15.104:6834/2089768912] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6832/2306780582,v1:172.21.15.178:6834/2306780582] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6833/2204870830,v1:172.21.15.178:6835/2204870830] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6836/199999798,v1:172.21.15.178:6837/199999798] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.178:6838/3804315653,v1:172.21.15.178:6839/3804315653] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6836/99741491,v1:172.21.15.104:6838/99741491] up:boot 2024-04-25T13:15:34.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: mds.? [v2:172.21.15.104:6837/3835401869,v1:172.21.15.104:6839/3835401869] up:boot 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: fsmap 12 up:standby 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:15:34.328 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:15:34.329 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:15:34.329 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:33 smithi104 ceph-mon[32726]: from='mgr.15345 172.21.15.38:0/2129661020' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:15:34.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.943+0000 7f8129ffb640 1 -- 172.21.15.38:0/160564945 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7f811c09c120 con 0x7f812c106160 2024-04-25T13:15:34.945 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:15:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f81180a3d50 msgr2=0x7f81180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f81180a3d50 0x7f81180a6210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8114009230 tx=0x7f81140058d0 comp rx=0 tx=0).stop 2024-04-25T13:15:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 msgr2=0x7f812c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 0x7f812c103060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f811c00d740 tx=0x7f811c00dc00 comp rx=0 tx=0).stop 2024-04-25T13:15:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 shutdown_connections 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f812c1035a0 0x7f812c1a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f812c106160 0x7f812c103060 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f81180a3d50 0x7f81180a6210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 --2- 172.21.15.38:0/160564945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f812c101090 0x7f812c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 >> 172.21.15.38:0/160564945 conn(0x7f812c077f50 msgr2=0x7f812c0694b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 shutdown_connections 2024-04-25T13:15:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:34.947+0000 7f8132fef640 1 -- 172.21.15.38:0/160564945 wait complete. 2024-04-25T13:15:34.960 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:15:35.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 -- 172.21.15.38:0/384329762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc107260 msgr2=0x7f77fc10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:35.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 --2- 172.21.15.38:0/384329762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc107260 0x7f77fc10ee30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009ca0 tx=0x7f77f402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 -- 172.21.15.38:0/384329762 shutdown_connections 2024-04-25T13:15:35.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 --2- 172.21.15.38:0/384329762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc107260 0x7f77fc10ee30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 --2- 172.21.15.38:0/384329762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77fc102b40 0x7f77fc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 --2- 172.21.15.38:0/384329762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77fc102190 0x7f77fc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 -- 172.21.15.38:0/384329762 >> 172.21.15.38:0/384329762 conn(0x7f77fc0fd990 msgr2=0x7f77fc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 -- 172.21.15.38:0/384329762 shutdown_connections 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 -- 172.21.15.38:0/384329762 wait complete. 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 Processor -- start 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.067+0000 7f7802cef640 1 -- start start 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 0x7f77fc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:35.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77fc107260 0x7f77fc19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77fc1985d0 0x7f77fc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc112350 con 0x7f77fc102190 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1121d0 con 0x7f77fc1985d0 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1124d0 con 0x7f77fc107260 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 0x7f77fc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 0x7f77fc19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56794/0 (socket says 172.21.15.38:56794) 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 -- 172.21.15.38:0/239484175 learned_addr learned my addr 172.21.15.38:0/239484175 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T13:15:35.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f780128c640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77fc1985d0 0x7f77fc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f77fbfff640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77fc107260 0x7f77fc19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 -- 172.21.15.38:0/239484175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77fc107260 msgr2=0x7f77fc19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77fc107260 0x7f77fc19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 -- 172.21.15.38:0/239484175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77fc1985d0 msgr2=0x7f77fc198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77fc1985d0 0x7f77fc198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 -- 172.21.15.38:0/239484175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1990f0 con 0x7f77fc102190 2024-04-25T13:15:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7800a8b640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 0x7f77fc19e420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c920 tx=0x7f77f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0007c70 con 0x7f77fc102190 2024-04-25T13:15:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1993e0 con 0x7f77fc102190 2024-04-25T13:15:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f77f0007e10 con 0x7f77fc102190 2024-04-25T13:15:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.068+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f00188c0 con 0x7f77fc102190 2024-04-25T13:15:35.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.069+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a5630 con 0x7f77fc102190 2024-04-25T13:15:35.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.069+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc0691c0 con 0x7f77fc102190 2024-04-25T13:15:35.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.070+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77f0018a60 con 0x7f77fc102190 2024-04-25T13:15:35.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.071+0000 7f77f9ffb640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:35.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.071+0000 7f77fbfff640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:35.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.071+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8403+0+0 (secure 0 0 0) 0x7f77f00d09f0 con 0x7f77fc102190 2024-04-25T13:15:35.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.072+0000 7f77fbfff640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77e8005fd0 tx=0x7f77e800b510 comp rx=0 tx=0).ready entity=mgr.15345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:35.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.076+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f0014030 con 0x7f77fc102190 2024-04-25T13:15:35.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:34 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/160564945' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:15:35.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.266+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77fc199b00 con 0x7f77fc102190 2024-04-25T13:15:35.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:34 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/160564945' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:15:35.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:34 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/160564945' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:15:35.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.945+0000 7f77f9ffb640 1 -- 172.21.15.38:0/239484175 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f77f000de20 con 0x7f77fc102190 2024-04-25T13:15:35.946 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T13:15:35.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f77ec0a3d20 msgr2=0x7f77ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:35.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77e8005fd0 tx=0x7f77e800b510 comp rx=0 tx=0).stop 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 msgr2=0x7f77fc19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 0x7f77fc19e420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c920 tx=0x7f77f000cde0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 shutdown_connections 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77fc1985d0 0x7f77fc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:6800/1710923873,v1:172.21.15.38:6801/1710923873] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77fc107260 0x7f77fc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 --2- 172.21.15.38:0/239484175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77fc102190 0x7f77fc19e420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.949+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 >> 172.21.15.38:0/239484175 conn(0x7f77fc0fd990 msgr2=0x7f77fc0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:35.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.950+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 shutdown_connections 2024-04-25T13:15:35.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T13:15:35.950+0000 7f7802cef640 1 -- 172.21.15.38:0/239484175 wait complete. 2024-04-25T13:15:35.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:15:35.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:15:36.030 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:15:36.065 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T13:15:36.066 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:15:36.083 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:15:36.103 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:15:36.121 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T13:15:36.121 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T13:15:36.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:15:36.240 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:35 smithi178 ceph-mon[33233]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 409 B/s wr, 8 op/s 2024-04-25T13:15:36.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:35 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/160564945' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:15:36.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:35 smithi178 ceph-mon[33233]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:15:36.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:35 smithi178 ceph-mon[33233]: from='client.? 172.21.15.38:0/239484175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:15:36.322 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:35 smithi038 ceph-mon[29400]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 409 B/s wr, 8 op/s 2024-04-25T13:15:36.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:35 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/160564945' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:15:36.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:35 smithi038 ceph-mon[29400]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:15:36.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:35 smithi038 ceph-mon[29400]: from='client.? 172.21.15.38:0/239484175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:15:36.323 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:36 smithi038 systemd[1]: Stopping Ceph mon.a for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:36.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:35 smithi104 ceph-mon[32726]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 409 B/s wr, 8 op/s 2024-04-25T13:15:36.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:35 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/160564945' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:15:36.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:35 smithi104 ceph-mon[32726]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:15:36.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:35 smithi104 ceph-mon[32726]: from='client.? 172.21.15.38:0/239484175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:15:36.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:36 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a[29396]: 2024-04-25T13:15:36.436+0000 7f4112056700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:36.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:36 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a[29396]: 2024-04-25T13:15:36.436+0000 7f4112056700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T13:15:36.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:36 smithi038 podman[84488]: 2024-04-25 13:15:36.673243538 +0000 UTC m=+0.350991935 container died 9c0a6e656491401ab3e9a055a8fbebadba01b84c116862882d1014210bd724bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:15:37.314 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:37 smithi038 podman[84488]: 2024-04-25 13:15:37.071776437 +0000 UTC m=+0.749524830 container cleanup 9c0a6e656491401ab3e9a055a8fbebadba01b84c116862882d1014210bd724bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T13:15:37.315 INFO:journalctl@ceph.mon.a.smithi038.stdout:Apr 25 13:15:37 smithi038 bash[84488]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:15:37.417 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T13:15:37.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:37.468 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T13:15:37.468 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T13:15:37.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.b 2024-04-25T13:15:37.775 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:37 smithi104 systemd[1]: Stopping Ceph mon.b for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-b[32722]: 2024-04-25T13:15:37.775+0000 7f9339727700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-b[32722]: 2024-04-25T13:15:37.775+0000 7f9339727700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:15:38.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:38 smithi104 podman[68636]: 2024-04-25 13:15:38.031080496 +0000 UTC m=+0.368571818 container died a92d3a01a46ac9fc39c65ecc81f8d0338e5ce677da061cf2fb35888514e7f363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T13:15:38.718 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:38 smithi104 podman[68636]: 2024-04-25 13:15:38.442919434 +0000 UTC m=+0.780410766 container cleanup a92d3a01a46ac9fc39c65ecc81f8d0338e5ce677da061cf2fb35888514e7f363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:15:38.719 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 25 13:15:38 smithi104 bash[68636]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-b 2024-04-25T13:15:38.874 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T13:15:38.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:38.922 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T13:15:38.922 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T13:15:38.922 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.c 2024-04-25T13:15:39.236 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:38 smithi178 systemd[1]: Stopping Ceph mon.c for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:39.490 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c[33207]: 2024-04-25T13:15:39.236+0000 7fe5d6b1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:39.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c[33207]: 2024-04-25T13:15:39.236+0000 7fe5d6b1c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:15:39.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:39 smithi178 podman[69706]: 2024-04-25 13:15:39.277616398 +0000 UTC m=+0.161073388 container died fd042fa8a8aaa8fe48546f7af2ab2f0d5363d8c8de3f287b47fc3897f8b7470e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-25T13:15:39.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:39 smithi178 podman[69706]: 2024-04-25 13:15:39.679558951 +0000 UTC m=+0.563015942 container cleanup fd042fa8a8aaa8fe48546f7af2ab2f0d5363d8c8de3f287b47fc3897f8b7470e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-25T13:15:39.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:39 smithi178 bash[69706]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c 2024-04-25T13:15:40.148 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T13:15:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:39 smithi178 podman[69720]: 2024-04-25 13:15:39.906963018 +0000 UTC m=+0.621879148 container remove fd042fa8a8aaa8fe48546f7af2ab2f0d5363d8c8de3f287b47fc3897f8b7470e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T13:15:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:40 smithi178 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-25T13:15:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:40 smithi178 systemd[1]: Stopped Ceph mon.c for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:15:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Apr 25 13:15:40 smithi178 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mon.c.service: Consumed 11.256s CPU time. 2024-04-25T13:15:40.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:40.195 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T13:15:40.196 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T13:15:40.196 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:15:40.526 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:40 smithi038 systemd[1]: Stopping Ceph mgr.x for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:40.822 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:40 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:15:40.525+0000 7f9e5763b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:40.823 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:40 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-25T13:15:40.525+0000 7f9e5763b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T13:15:40.823 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:40 smithi038 podman[84607]: 2024-04-25 13:15:40.557806159 +0000 UTC m=+0.156640047 container died e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:41.162 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:40 smithi038 podman[84607]: 2024-04-25 13:15:40.928021136 +0000 UTC m=+0.526855022 container cleanup e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T13:15:41.162 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:40 smithi038 bash[84607]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x 2024-04-25T13:15:41.408 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T13:15:41.430 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:41 smithi038 podman[84620]: 2024-04-25 13:15:41.162969958 +0000 UTC m=+0.602233401 container remove e85391fcd6846b75d0836674125a0f574e25c8c603c26af85ee7462b67625039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T13:15:41.430 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:41 smithi038 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-25T13:15:41.430 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:41 smithi038 systemd[1]: Stopped Ceph mgr.x for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:15:41.431 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Apr 25 13:15:41 smithi038 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.x.service: Consumed 1min 29.138s CPU time. 2024-04-25T13:15:41.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:41.444 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T13:15:41.445 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T13:15:41.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.y 2024-04-25T13:15:41.767 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:15:41 smithi104 systemd[1]: Stopping Ceph mgr.y for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:42.075 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:15:41 smithi104 podman[68758]: 2024-04-25 13:15:41.768213784 +0000 UTC m=+0.136652919 container died f44d51b5520cae9dc9ffdc83b29a64f2e2566c0ed525b291c2b59214a57aaa65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:15:42.407 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:15:42 smithi104 podman[68758]: 2024-04-25 13:15:42.148008887 +0000 UTC m=+0.516448019 container cleanup f44d51b5520cae9dc9ffdc83b29a64f2e2566c0ed525b291c2b59214a57aaa65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:15:42.407 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 25 13:15:42 smithi104 bash[68758]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-y 2024-04-25T13:15:42.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T13:15:42.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:42.561 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T13:15:42.562 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T13:15:42.562 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.z 2024-04-25T13:15:42.924 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:15:42 smithi178 systemd[1]: Stopping Ceph mgr.z for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:43.202 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:15:42 smithi178 podman[69829]: 2024-04-25 13:15:42.924412547 +0000 UTC m=+0.164222966 container died 07d8327823261f08e816cfbde1cb713a4481817f3cb5feafc9b82e0ffab8e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:15:43.490 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:15:43 smithi178 podman[69829]: 2024-04-25 13:15:43.286055313 +0000 UTC m=+0.525865726 container cleanup 07d8327823261f08e816cfbde1cb713a4481817f3cb5feafc9b82e0ffab8e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T13:15:43.490 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Apr 25 13:15:43 smithi178 bash[69829]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mgr-z 2024-04-25T13:15:43.711 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T13:15:43.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:43.745 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T13:15:43.745 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T13:15:43.745 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.0 2024-04-25T13:15:44.072 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:43 smithi038 systemd[1]: Stopping Ceph osd.0 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:44.572 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0[41521]: 2024-04-25T13:15:44.224+0000 7f526eb92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:44.573 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0[41521]: 2024-04-25T13:15:44.224+0000 7f526eb92700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-25T13:15:44.573 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0[41521]: 2024-04-25T13:15:44.224+0000 7f526eb92700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:15:50.010 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:49 smithi038 podman[84728]: 2024-04-25 13:15:49.58560052 +0000 UTC m=+5.640275934 container died 7a23938e979ad2e06926c3f95b48c829ec1ef03a34b94155c25ffe3a60e71e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T13:15:50.310 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 podman[84728]: 2024-04-25 13:15:50.083895666 +0000 UTC m=+6.138571069 container cleanup 7a23938e979ad2e06926c3f95b48c829ec1ef03a34b94155c25ffe3a60e71e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:15:50.310 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 bash[84728]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0 2024-04-25T13:15:50.573 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 podman[84744]: 2024-04-25 13:15:50.310573637 +0000 UTC m=+0.726565195 container remove 7a23938e979ad2e06926c3f95b48c829ec1ef03a34b94155c25ffe3a60e71e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T13:15:50.897 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 podman[84835]: 2024-04-25 13:15:50.542897187 +0000 UTC m=+0.018637980 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:50.897 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 podman[84835]: 2024-04-25 13:15:50.644938581 +0000 UTC m=+0.120679373 container create 63d9854c3e9c8785544591b2f1a8ac1b0603dbc9f8c7548ea74e86225bb00bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T13:15:50.897 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 podman[84835]: 2024-04-25 13:15:50.897895703 +0000 UTC m=+0.373636496 container init 63d9854c3e9c8785544591b2f1a8ac1b0603dbc9f8c7548ea74e86225bb00bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T13:15:51.202 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:50 smithi038 podman[84835]: 2024-04-25 13:15:50.901065655 +0000 UTC m=+0.376806454 container start 63d9854c3e9c8785544591b2f1a8ac1b0603dbc9f8c7548ea74e86225bb00bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:15:51.203 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:51 smithi038 podman[84835]: 2024-04-25 13:15:51.006491853 +0000 UTC m=+0.482232644 container attach 63d9854c3e9c8785544591b2f1a8ac1b0603dbc9f8c7548ea74e86225bb00bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T13:15:51.573 INFO:journalctl@ceph.osd.0.smithi038.stdout:Apr 25 13:15:51 smithi038 podman[84835]: 2024-04-25 13:15:51.202348076 +0000 UTC m=+0.678088875 container died 63d9854c3e9c8785544591b2f1a8ac1b0603dbc9f8c7548ea74e86225bb00bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:15:51.998 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T13:15:52.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:52.043 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T13:15:52.044 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T13:15:52.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.1 2024-04-25T13:15:52.325 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:52 smithi104 systemd[1]: Stopping Ceph osd.1 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:15:52.825 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1[37344]: 2024-04-25T13:15:52.367+0000 7f89ca96f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:15:52.825 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1[37344]: 2024-04-25T13:15:52.367+0000 7f89ca96f700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-25T13:15:52.825 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1[37344]: 2024-04-25T13:15:52.367+0000 7f89ca96f700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:15:58.126 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:57 smithi104 podman[68887]: 2024-04-25 13:15:57.833130635 +0000 UTC m=+5.590286443 container died f833e7711f31ec45a6200ca0e794b296f6c51e2951eb4dd753afca6d1b133326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T13:15:58.494 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:58 smithi104 podman[68887]: 2024-04-25 13:15:58.234482729 +0000 UTC m=+5.991638536 container cleanup f833e7711f31ec45a6200ca0e794b296f6c51e2951eb4dd753afca6d1b133326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:15:58.494 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:58 smithi104 bash[68887]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1 2024-04-25T13:15:58.825 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:58 smithi104 podman[68904]: 2024-04-25 13:15:58.494273361 +0000 UTC m=+0.657544518 container remove f833e7711f31ec45a6200ca0e794b296f6c51e2951eb4dd753afca6d1b133326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:15:59.197 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:58 smithi104 podman[68991]: 2024-04-25 13:15:58.755075137 +0000 UTC m=+0.017964905 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:15:59.197 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:58 smithi104 podman[68991]: 2024-04-25 13:15:58.853393343 +0000 UTC m=+0.116283112 container create 453443b0db19f784726e16b01fa8a603889123a892cc42c542c3abdd471e3ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T13:15:59.197 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:59 smithi104 podman[68991]: 2024-04-25 13:15:59.089291125 +0000 UTC m=+0.352180894 container init 453443b0db19f784726e16b01fa8a603889123a892cc42c542c3abdd471e3ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T13:15:59.197 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:59 smithi104 podman[68991]: 2024-04-25 13:15:59.092572659 +0000 UTC m=+0.355462428 container start 453443b0db19f784726e16b01fa8a603889123a892cc42c542c3abdd471e3ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T13:15:59.198 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:59 smithi104 podman[68991]: 2024-04-25 13:15:59.197692913 +0000 UTC m=+0.460582703 container attach 453443b0db19f784726e16b01fa8a603889123a892cc42c542c3abdd471e3ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:15:59.575 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 25 13:15:59 smithi104 podman[68991]: 2024-04-25 13:15:59.37692659 +0000 UTC m=+0.639816358 container died 453443b0db19f784726e16b01fa8a603889123a892cc42c542c3abdd471e3ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:15:59.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T13:15:59.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:15:59.975 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T13:15:59.975 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T13:15:59.975 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.2 2024-04-25T13:16:00.268 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:00 smithi178 systemd[1]: Stopping Ceph osd.2 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:00.675 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2[38383]: 2024-04-25T13:16:00.306+0000 7fa9ffbea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:00.675 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2[38383]: 2024-04-25T13:16:00.306+0000 7fa9ffbea700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-25T13:16:00.676 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2[38383]: 2024-04-25T13:16:00.306+0000 7fa9ffbea700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:05.975 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:05 smithi178 podman[69958]: 2024-04-25 13:16:05.689082737 +0000 UTC m=+5.509573459 container died 227864c65b63dc3232d26b3ca4cd0bf8997d12ae4fb4cd50273f5cfb1cd0c077 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:16:06.240 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 podman[69958]: 2024-04-25 13:16:06.074982264 +0000 UTC m=+5.895472980 container cleanup 227864c65b63dc3232d26b3ca4cd0bf8997d12ae4fb4cd50273f5cfb1cd0c077 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:16:06.240 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 bash[69958]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2 2024-04-25T13:16:06.704 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 podman[69987]: 2024-04-25 13:16:06.343188982 +0000 UTC m=+0.649429300 container remove 227864c65b63dc3232d26b3ca4cd0bf8997d12ae4fb4cd50273f5cfb1cd0c077 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:16:06.988 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 podman[70079]: 2024-04-25 13:16:06.608134282 +0000 UTC m=+0.018160391 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:06.988 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 podman[70079]: 2024-04-25 13:16:06.727776348 +0000 UTC m=+0.137802444 container create 41326cd0e2e92998d0f4de5c20ca7e3f8cf9f3287de5a88a5dba437bd38b8cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T13:16:06.988 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 podman[70079]: 2024-04-25 13:16:06.989047825 +0000 UTC m=+0.399073918 container init 41326cd0e2e92998d0f4de5c20ca7e3f8cf9f3287de5a88a5dba437bd38b8cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:16:07.240 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:06 smithi178 podman[70079]: 2024-04-25 13:16:06.992117686 +0000 UTC m=+0.402143784 container start 41326cd0e2e92998d0f4de5c20ca7e3f8cf9f3287de5a88a5dba437bd38b8cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:07.241 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:07 smithi178 podman[70079]: 2024-04-25 13:16:07.080808009 +0000 UTC m=+0.490834121 container attach 41326cd0e2e92998d0f4de5c20ca7e3f8cf9f3287de5a88a5dba437bd38b8cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T13:16:07.628 INFO:journalctl@ceph.osd.2.smithi178.stdout:Apr 25 13:16:07 smithi178 podman[70079]: 2024-04-25 13:16:07.279272106 +0000 UTC m=+0.689298205 container died 41326cd0e2e92998d0f4de5c20ca7e3f8cf9f3287de5a88a5dba437bd38b8cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T13:16:07.974 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T13:16:08.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:08.012 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T13:16:08.012 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T13:16:08.013 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.3 2024-04-25T13:16:08.322 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:08 smithi038 systemd[1]: Stopping Ceph osd.3 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:08.616 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:08 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3[48060]: 2024-04-25T13:16:08.466+0000 7f0b41c8d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:08.616 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:08 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3[48060]: 2024-04-25T13:16:08.466+0000 7f0b41c8d700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-25T13:16:08.617 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:08 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3[48060]: 2024-04-25T13:16:08.466+0000 7f0b41c8d700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:10.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:10 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:10.460+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:10.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:10.716+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:11.575 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:11 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:11.463+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:11.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:11.703+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:11.753+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:12.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:12.189+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:12.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:12 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:12.459+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:12.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:12.728+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:12.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:12.708+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:13.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:12 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:12.753+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:13.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:13 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:13.197+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:13.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:13 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:13.421+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:13.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:13.713+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:13.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:13.671+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:14.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:13 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:13.794+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:14.073 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:13 smithi038 podman[84942]: 2024-04-25 13:16:13.886731086 +0000 UTC m=+5.671644481 container died 95d4bb102f1f72103ceb0859f17732ff3af06d89e087c3085403eaf5d8ae07d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T13:16:14.573 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:14 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:14.214+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:14.573 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:14 smithi038 podman[84942]: 2024-04-25 13:16:14.364203071 +0000 UTC m=+6.149116461 container cleanup 95d4bb102f1f72103ceb0859f17732ff3af06d89e087c3085403eaf5d8ae07d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T13:16:14.573 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:14 smithi038 bash[84942]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3 2024-04-25T13:16:14.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:14 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:14.397+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:14.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:14.731+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:14.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:14.646+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:15.044 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:14 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:14.757+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:15.044 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:14 smithi038 podman[84970]: 2024-04-25 13:16:14.690833558 +0000 UTC m=+0.796813099 container remove 95d4bb102f1f72103ceb0859f17732ff3af06d89e087c3085403eaf5d8ae07d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:16:15.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:15 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:15.179+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:15.323 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:15 smithi038 podman[85061]: 2024-04-25 13:16:14.948422992 +0000 UTC m=+0.018513621 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:15.323 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:15 smithi038 podman[85061]: 2024-04-25 13:16:15.058569313 +0000 UTC m=+0.128659928 container create ed7454b4838268a043a33206f24b8c985efdd59d24a1ab211044e41b0f87d13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:16:15.323 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:15 smithi038 podman[85061]: 2024-04-25 13:16:15.277815392 +0000 UTC m=+0.347906001 container init ed7454b4838268a043a33206f24b8c985efdd59d24a1ab211044e41b0f87d13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T13:16:15.323 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:15 smithi038 podman[85061]: 2024-04-25 13:16:15.280986408 +0000 UTC m=+0.351077024 container start ed7454b4838268a043a33206f24b8c985efdd59d24a1ab211044e41b0f87d13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T13:16:15.588 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:15 smithi038 podman[85061]: 2024-04-25 13:16:15.428521273 +0000 UTC m=+0.498611898 container attach ed7454b4838268a043a33206f24b8c985efdd59d24a1ab211044e41b0f87d13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T13:16:15.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:15 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:15.423+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:15.931 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:15 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:15.720+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:15.932 INFO:journalctl@ceph.osd.3.smithi038.stdout:Apr 25 13:16:15 smithi038 podman[85061]: 2024-04-25 13:16:15.589248424 +0000 UTC m=+0.659339036 container died ed7454b4838268a043a33206f24b8c985efdd59d24a1ab211044e41b0f87d13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:16:15.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:15.712+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:15.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:15.628+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:16.198 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:16 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:16.172+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:16.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T13:16:16.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:16.258 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T13:16:16.258 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T13:16:16.259 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.4 2024-04-25T13:16:16.569 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:16 smithi104 systemd[1]: Stopping Ceph osd.4 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:16.570 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:16 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:16.446+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:16.825 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:16 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T13:16:16.568+0000 7fcecc094700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:16.826 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:16 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T13:16:16.568+0000 7fcecc094700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-25T13:16:16.826 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:16 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T13:16:16.568+0000 7fcecc094700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:16.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:16 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:16.724+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:16.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:16 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:16.606+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:17.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:16 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:16.752+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:17.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:17 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:17.219+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:17.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:17 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:17.452+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:17.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:17 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:17.690+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:17.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:17 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:17.598+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:18.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:17 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:17.759+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:18.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:18 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:18.251+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:18.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:18 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:18.479+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:18.990 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:18 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:18.734+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:18.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:18 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:18.593+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:19.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:18 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:18.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:19.325 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:18 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T13:16:18.995+0000 7fcec8688700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:52.694840+0000 front 2024-04-25T13:15:52.694758+0000 (oldest deadline 2024-04-25T13:16:18.495364+0000) 2024-04-25T13:16:19.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:19 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:19.230+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:19.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:19 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:19.472+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:19.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:19 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:19.472+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:19.953 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:19 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:19.599+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:19.954 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:19 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:19.754+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:20.205 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:19 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:19.854+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:20.205 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:19 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:19.854+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:20.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:19 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:19.954+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:20.325 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:19 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T13:16:19.973+0000 7fcec8688700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:52.694840+0000 front 2024-04-25T13:15:52.694758+0000 (oldest deadline 2024-04-25T13:16:18.495364+0000) 2024-04-25T13:16:20.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:20 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:20.206+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:20.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:20 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:20.548+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:20.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:20 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:20.509+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:20.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:20 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:20.509+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:20.920 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:20 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:20.738+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:20.920 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:20 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:20.601+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:21.180 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:20 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:20.841+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:21.180 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:20 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:20.841+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:21.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:20 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:20.920+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:21.325 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:20 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4[44281]: 2024-04-25T13:16:20.967+0000 7fcec8688700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:52.694840+0000 front 2024-04-25T13:15:52.694758+0000 (oldest deadline 2024-04-25T13:16:18.495364+0000) 2024-04-25T13:16:21.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:21 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:21.180+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:21.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:21 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:21.579+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:21.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:21 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:21.541+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:21.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:21 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:21.541+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:21.939 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:21 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:21.611+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:21.939 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:21 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:21.757+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:21.940 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:21 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:21.757+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:22.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:21 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:21.939+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:22.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:21 smithi104 podman[69107]: 2024-04-25 13:16:21.982114618 +0000 UTC m=+5.531803403 container died 45cdd57061e10e84df49ec526f7117a43f56544b58b98c8892b587f7894a07e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:22.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:21 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:21.890+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:22.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:21 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:21.890+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:22.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:22 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:22.135+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:22.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:22 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:22.135+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:22.533 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:22 smithi104 podman[69107]: 2024-04-25 13:16:22.36939115 +0000 UTC m=+5.919079967 container cleanup 45cdd57061e10e84df49ec526f7117a43f56544b58b98c8892b587f7894a07e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T13:16:22.533 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:22 smithi104 bash[69107]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4 2024-04-25T13:16:22.825 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:22 smithi104 podman[69126]: 2024-04-25 13:16:22.570667931 +0000 UTC m=+0.588063402 container remove 45cdd57061e10e84df49ec526f7117a43f56544b58b98c8892b587f7894a07e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T13:16:22.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:22 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:22.533+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:22.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:22 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:22.557+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:22.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:22 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:22.557+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:22.947 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:22 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:22.568+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:22.948 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:22 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:22.748+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:22.948 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:22 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:22.748+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:23.155 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:22 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:22.871+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:23.156 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:22 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:22.871+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:23.231 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:22 smithi104 podman[69214]: 2024-04-25 13:16:22.883620967 +0000 UTC m=+0.018304401 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:23.232 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:22 smithi104 podman[69214]: 2024-04-25 13:16:22.988397463 +0000 UTC m=+0.123080928 container create d54d87f81fdeccce45e539ba7e3e8fa46020b9dd33bbec33f81a054e45b9ac2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:23.232 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:23 smithi104 podman[69214]: 2024-04-25 13:16:23.232382796 +0000 UTC m=+0.367066217 container init d54d87f81fdeccce45e539ba7e3e8fa46020b9dd33bbec33f81a054e45b9ac2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:16:23.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:22 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:22.948+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:23.528 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:23 smithi104 podman[69214]: 2024-04-25 13:16:23.23549866 +0000 UTC m=+0.370182081 container start d54d87f81fdeccce45e539ba7e3e8fa46020b9dd33bbec33f81a054e45b9ac2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T13:16:23.528 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:23 smithi104 podman[69214]: 2024-04-25 13:16:23.340940791 +0000 UTC m=+0.475624212 container attach d54d87f81fdeccce45e539ba7e3e8fa46020b9dd33bbec33f81a054e45b9ac2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:16:23.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:23 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:23.155+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:23.573 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:23 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:23.155+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:23.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:23 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:23.535+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:23.825 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 25 13:16:23 smithi104 podman[69214]: 2024-04-25 13:16:23.533457022 +0000 UTC m=+0.668140442 container died d54d87f81fdeccce45e539ba7e3e8fa46020b9dd33bbec33f81a054e45b9ac2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T13:16:23.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:23 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:23.527+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:23.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:23 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:23.527+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:23.925 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:23 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:23.797+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:23.925 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:23 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:23.797+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:23 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:23.560+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:23 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:23.560+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:24.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T13:16:24.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:24.213 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T13:16:24.213 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T13:16:24.213 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.5 2024-04-25T13:16:24.238 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:23 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:23.925+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:24.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:23 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:23.909+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:24.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:23 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:23.909+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:24.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:24 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:24.143+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:24.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:24 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:24.143+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:24.490 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:24 smithi178 systemd[1]: Stopping Ceph osd.5 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:24.508 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:24 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:24.508+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:24.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:24 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:24.552+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:24.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:24 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:24.552+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:24.843 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:24.525+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:24.843 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:24.525+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:24.843 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:24.538+0000 7f4221a1e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:24.843 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:24.538+0000 7f4221a1e700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-25T13:16:24.843 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:24.538+0000 7f4221a1e700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:25.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:24.911+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:25.240 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:24.843+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:25.241 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:24 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:24.843+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:25.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:25 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:25.151+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:25.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:25 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:25.151+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:25.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:24 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:24.937+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:25.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:24 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:24.937+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:25.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:25 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:25.535+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:25.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:25 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:25.576+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:25.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:25 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:25.576+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:25.867 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:25 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:25.560+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:25.867 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:25 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:25.560+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:26.240 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:25 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:25.867+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:26.240 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:25 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:25.867+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:26.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:25 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:25.951+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:26.269 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:25 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:25.974+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:26.269 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:25 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:25.974+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:26.269 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:26 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:26.117+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:26.269 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:26 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:26.117+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:26.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:26 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:26.489+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:26.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:26 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:26.617+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:26.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:26 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:26.617+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:26.871 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:26.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:26.871 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:26.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:26.871 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:26.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:27.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:26.913+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:27.241 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:26.870+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:27.241 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:26.870+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:27.241 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:26 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:26.870+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.850352+0000 front 2024-04-25T13:16:03.850171+0000 (oldest deadline 2024-04-25T13:16:26.150325+0000) 2024-04-25T13:16:27.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:27.138+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:27.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:27.138+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:27.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:27.017+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:27.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:27.017+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:27.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:27.017+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:27.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:27 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:27.502+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:27.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:27 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:27.618+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:27.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:27 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:27.618+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:27.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:27 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:27.618+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:27.906 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:27.856+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:27.906 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:27.856+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:27.906 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:27.856+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.850352+0000 front 2024-04-25T13:16:03.850171+0000 (oldest deadline 2024-04-25T13:16:26.150325+0000) 2024-04-25T13:16:27.907 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:27.618+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:27.907 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:27.618+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:27.907 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:27.618+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:28.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:27 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:27.905+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:28.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:28.126+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:28.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:28.126+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:28.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:27.985+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:28.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:27.985+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:28.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:27.985+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:28.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:28 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:28.524+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:28.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:28 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:28.636+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:28.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:28 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:28.636+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:28.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:28 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:28.636+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:28.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:28.872+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:28.991 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:28.851+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:45.446041+0000 front 2024-04-25T13:15:45.446132+0000 (oldest deadline 2024-04-25T13:16:10.145918+0000) 2024-04-25T13:16:28.991 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:28.851+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.147494+0000 front 2024-04-25T13:15:57.147553+0000 (oldest deadline 2024-04-25T13:16:21.148285+0000) 2024-04-25T13:16:28.991 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5[45209]: 2024-04-25T13:16:28.851+0000 7f421a00a700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.850352+0000 front 2024-04-25T13:16:03.850171+0000 (oldest deadline 2024-04-25T13:16:26.150325+0000) 2024-04-25T13:16:28.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:28.635+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:28.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:28.635+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:28.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:28 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:28.635+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:29.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:29.161+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:29.324 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:29.161+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:29.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:28.940+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:29.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:28.940+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:29.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:28 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:28.940+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:29.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:29 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:29.605+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:29.827 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:29 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:29.605+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:29.827 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:29 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:29.605+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:29.827 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:29 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:29.570+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:29.827 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:29 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:29.570+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:29.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:29 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:29.612+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:29.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:29 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:29.612+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:29.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:29 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:29.612+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:30.241 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:29 smithi178 podman[70184]: 2024-04-25 13:16:29.903837374 +0000 UTC m=+5.498627577 container died 28e595671c81475776ecce67c66e87c30b0b8cf93f9bafc47d9e3ddff37378fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T13:16:30.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:29 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:29.897+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:30.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:29 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:29.897+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:30.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:30.114+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:30.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:30.114+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:30.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:29.909+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:30.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:29.909+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:30.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:29.909+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:30.592 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:30 smithi178 podman[70184]: 2024-04-25 13:16:30.356121569 +0000 UTC m=+5.950911811 container cleanup 28e595671c81475776ecce67c66e87c30b0b8cf93f9bafc47d9e3ddff37378fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T13:16:30.592 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:30 smithi178 bash[70184]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5 2024-04-25T13:16:30.942 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:30 smithi178 podman[70199]: 2024-04-25 13:16:30.641182051 +0000 UTC m=+0.737896890 container remove 28e595671c81475776ecce67c66e87c30b0b8cf93f9bafc47d9e3ddff37378fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T13:16:30.943 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:30 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:30.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:30.943 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:30 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:30.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:30.943 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:30 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:30.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:31.075 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:30 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:30.604+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:31.075 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:30 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:30.604+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:31.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:30 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:30.629+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:31.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:30 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:30.629+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:31.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:30 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:30.629+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:31.240 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:30 smithi178 podman[70292]: 2024-04-25 13:16:30.872602188 +0000 UTC m=+0.017814236 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:31.241 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:30 smithi178 podman[70292]: 2024-04-25 13:16:30.992182726 +0000 UTC m=+0.137394747 container create 00a37410e525806a2470ee5f1246ccb96166e8f5702775cc825da2f068f43b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:16:31.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:30 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:30.943+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:31.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:30 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:30.943+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:31.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:31.091+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:31.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:31.091+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:31.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:31.092+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:31.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:30.884+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:31.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:30.884+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:31.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:30 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:30.884+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:31.541 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:31 smithi178 podman[70292]: 2024-04-25 13:16:31.245443805 +0000 UTC m=+0.390655832 container init 00a37410e525806a2470ee5f1246ccb96166e8f5702775cc825da2f068f43b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:16:31.541 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:31 smithi178 podman[70292]: 2024-04-25 13:16:31.249481108 +0000 UTC m=+0.394693131 container start 00a37410e525806a2470ee5f1246ccb96166e8f5702775cc825da2f068f43b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T13:16:31.541 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:31 smithi178 podman[70292]: 2024-04-25 13:16:31.345684569 +0000 UTC m=+0.490896591 container attach 00a37410e525806a2470ee5f1246ccb96166e8f5702775cc825da2f068f43b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T13:16:31.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:31 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:31.564+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:31.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:31 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:31.564+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:31.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:31 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:31.677+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:31.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:31 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:31.677+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:31.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:31 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:31.677+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:31.847 INFO:journalctl@ceph.osd.5.smithi178.stdout:Apr 25 13:16:31 smithi178 podman[70292]: 2024-04-25 13:16:31.541866138 +0000 UTC m=+0.687078171 container died 00a37410e525806a2470ee5f1246ccb96166e8f5702775cc825da2f068f43b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T13:16:31.848 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:31 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:31.590+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:31.848 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:31 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:31.590+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:31.848 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:31 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:31.590+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:32.126 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:31 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:31.957+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:32.127 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:31 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:31.957+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:32.143 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T13:16:32.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:32.223 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T13:16:32.223 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T13:16:32.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.6 2024-04-25T13:16:32.251 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:31.884+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:32.251 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:31.884+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:32.251 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:31 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:31.884+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:32.251 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:32.113+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:32.251 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:32.113+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:32.251 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:32.113+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:32.551 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 systemd[1]: Stopping Ceph osd.6 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:32.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:32.611+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:32.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:32.611+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:32.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:32.611+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:32.822 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:32.550+0000 7fd199ffc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:32.823 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:32.550+0000 7fd199ffc700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-25T13:16:32.823 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:32.550+0000 7fd199ffc700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:32.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:32.522+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:32.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:32.522+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:32.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:32.685+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:32.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:32.685+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:32.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:32.685+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:33.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:32.924+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:33.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:32 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:32.924+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:33.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:33.076+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:33.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:33.076+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:33.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:33.076+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:33.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:32.846+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:33.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:32.846+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:33.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:32 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:32.846+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:33.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:33 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:33.555+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:33.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:33 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:33.555+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:33.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:33 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:33.719+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:33.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:33 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:33.720+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:33.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:33 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:33.720+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:33.910 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:33 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:33.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:33.911 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:33 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:33.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:33.911 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:33 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:33.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:34.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:33 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:33.911+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:34.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:33 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:33.911+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:34.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:34.074+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:34.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:34.074+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:34.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:34.074+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:34.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:33.863+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:34.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:33.863+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:34.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:33 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:33.863+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:34.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:34 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:34.514+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:34.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:34 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:34.514+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:34.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:34 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:34.720+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:34.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:34 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:34.720+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:34.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:34 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:34.720+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:34.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:34 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:34.720+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:34.890 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:34 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:34.629+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:34.890 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:34 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:34.629+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:34.890 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:34 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:34.629+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:35.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:34 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:34.890+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:35.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:34 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:34.890+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:35.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.038+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:35.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.038+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:35.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.038+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:35.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:34.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:35.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:34.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:35.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:34 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:34.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:35.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:35.510+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:35.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:35.510+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:35.826 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:35.510+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:35.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:35.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:35.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:35.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:35.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:35.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:35.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:35 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:35.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:35.922 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:35.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:35.922 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:35.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:35.922 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:35.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:35.922 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:35.602+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:36.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:35.922+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:36.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:35 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:35.922+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:36.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.991+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.991+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.991+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:35.991+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:13.418705+0000 front 2024-04-25T13:16:13.418772+0000 (oldest deadline 2024-04-25T13:16:35.718968+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:35.875+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:35.875+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:35.875+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:36.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:35 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:35.875+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:36.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:36.546+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:36.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:36.546+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:36.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:36.546+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:36.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:36.651+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:36.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:36.651+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:36.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:36.651+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:36.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:36 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:36.651+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:36.873 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:36 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:36.572+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:36.874 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:36 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:36.572+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:36.874 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:36 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:36.572+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:36.874 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:36 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:36.572+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:37.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:36 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:36.873+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:37.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:36 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:36.873+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:37.322 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:37.003+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:48.513208+0000 front 2024-04-25T13:15:48.513234+0000 (oldest deadline 2024-04-25T13:16:11.413392+0000) 2024-04-25T13:16:37.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:37.003+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.215163+0000 front 2024-04-25T13:15:57.215109+0000 (oldest deadline 2024-04-25T13:16:21.315197+0000) 2024-04-25T13:16:37.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:37.003+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.816284+0000 front 2024-04-25T13:16:04.816430+0000 (oldest deadline 2024-04-25T13:16:30.116351+0000) 2024-04-25T13:16:37.323 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6[54577]: 2024-04-25T13:16:37.003+0000 7fd1925e8700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:13.418705+0000 front 2024-04-25T13:16:13.418772+0000 (oldest deadline 2024-04-25T13:16:35.718968+0000) 2024-04-25T13:16:37.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:36 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:36.862+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:37.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:36 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:36.862+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:37.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:36 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:36.862+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:37.324 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:36 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:36.862+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:37.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:37.510+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:37.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:37.510+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:37.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:37.510+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:37.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:37.664+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:37.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:37.664+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:37.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:37.664+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:37.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:37.664+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:37.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:37.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:37.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:37.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:37.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:37.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:37.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:37.592+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:38.160 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:37 smithi038 podman[85169]: 2024-04-25 13:16:37.868109922 +0000 UTC m=+5.441361897 container died e28f69ffbbed347b30b3ce4e7bdb9ced32cf894ec53aff176dd4511fd4de2b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:16:38.161 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:37.910+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:38.161 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:37.910+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:38.161 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:37.910+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:38.161 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:37 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:37.910+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:38.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:37.896+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:38.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:37.896+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:38.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:37 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:37.896+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:38.462 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:38 smithi038 podman[85169]: 2024-04-25 13:16:38.25230403 +0000 UTC m=+5.825556014 container cleanup e28f69ffbbed347b30b3ce4e7bdb9ced32cf894ec53aff176dd4511fd4de2b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:16:38.462 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:38 smithi038 bash[85169]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6 2024-04-25T13:16:38.794 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:38 smithi038 podman[85186]: 2024-04-25 13:16:38.461649872 +0000 UTC m=+0.587797862 container remove e28f69ffbbed347b30b3ce4e7bdb9ced32cf894ec53aff176dd4511fd4de2b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T13:16:38.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:38.502+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:38.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:38.502+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:38.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:38.502+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:38.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:38.615+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:38.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:38.615+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:38.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:38.615+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:38.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:38 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:38.615+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:38.854 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:38.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:38.854 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:38.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:38.854 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:38.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:38.854 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:38.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:39.073 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:38 smithi038 podman[85278]: 2024-04-25 13:16:38.698059892 +0000 UTC m=+0.018302298 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:39.073 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:38 smithi038 podman[85278]: 2024-04-25 13:16:38.821020918 +0000 UTC m=+0.141263342 container create fac8232abffcba2d6a4fb83b55ba67c690c087d8e0d7aa0ad9279657df509d26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T13:16:39.073 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:39 smithi038 podman[85278]: 2024-04-25 13:16:39.015630322 +0000 UTC m=+0.335872712 container init fac8232abffcba2d6a4fb83b55ba67c690c087d8e0d7aa0ad9279657df509d26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:16:39.073 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:39 smithi038 podman[85278]: 2024-04-25 13:16:39.019080396 +0000 UTC m=+0.339322785 container start fac8232abffcba2d6a4fb83b55ba67c690c087d8e0d7aa0ad9279657df509d26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:16:39.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:38 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:38.918+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:39.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:38 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:38.918+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:39.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:38 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:38.918+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:39.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:38 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:38.918+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:39.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:38.853+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:39.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:38.853+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:39.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:38 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:38.853+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:39.572 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:39 smithi038 podman[85278]: 2024-04-25 13:16:39.081650048 +0000 UTC m=+0.401892430 container attach fac8232abffcba2d6a4fb83b55ba67c690c087d8e0d7aa0ad9279657df509d26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T13:16:39.573 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:39 smithi038 podman[85278]: 2024-04-25 13:16:39.323476667 +0000 UTC m=+0.643719051 container died fac8232abffcba2d6a4fb83b55ba67c690c087d8e0d7aa0ad9279657df509d26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:16:39.810 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:39.553+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:39.811 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:39.553+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:39.811 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:39.553+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:39.811 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:39.553+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:39.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:39.493+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:39.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:39.493+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:39.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:39.493+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:39.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:39.621+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:39.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:39.621+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:39.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:39.621+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:39.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:39 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:39.621+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:39.877 INFO:journalctl@ceph.osd.6.smithi038.stdout:Apr 25 13:16:39 smithi038 podman[85304]: 2024-04-25 13:16:39.869043491 +0000 UTC m=+0.535831941 container remove fac8232abffcba2d6a4fb83b55ba67c690c087d8e0d7aa0ad9279657df509d26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T13:16:39.894 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T13:16:39.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:39.976 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T13:16:39.976 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T13:16:39.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.7 2024-04-25T13:16:40.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:39.810+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:40.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:39.810+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:40.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:39 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:39.810+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:40.287 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 systemd[1]: Stopping Ceph osd.7 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:40.292 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:39 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:39.877+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:40.292 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:39 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:39.877+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:40.293 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:39 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:39.877+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:40.293 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:39 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:39.877+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:40.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:40.287+0000 7f5bc29a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:40.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:40.287+0000 7f5bc29a1700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-25T13:16:40.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:40.287+0000 7f5bc29a1700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:40.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:40.519+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:40.576 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:40.519+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:40.576 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:40.519+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:40.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:40.575+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:40.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:40.575+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:40.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:40.575+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:40.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:40.575+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:41.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:40.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:41.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:40.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:41.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:40.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:41.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:40 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:40.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:41.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:40.840+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:41.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:40.840+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:41.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:40 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:40.840+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:41.264 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:40 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:40.919+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:41.264 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:40 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:40.919+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:41.264 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:40 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:40.919+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:41.265 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:40 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:40.919+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:41.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:41.543+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:41.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:41.543+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:41.826 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:41.543+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:41.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:41.619+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:41.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:41.619+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:41.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:41.619+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:41.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:41.619+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:41.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:41.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:41.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:41.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:41.841 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:41.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:41.842 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:41.586+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:42.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:41.841+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:42.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:41.841+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:42.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:41 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:41.841+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:42.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:41.924+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:42.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:41.924+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:42.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:41.924+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:42.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:41 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:41.924+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:42.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:42.533+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:42.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:42.533+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:42.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:42.533+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:42.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:42.648+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:42.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:42.648+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:42.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:42.648+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:42.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:42 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:42.648+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:42.838 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:42.818+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:42.838 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:42.818+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:42.838 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:42.818+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:42.839 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:42.587+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:42.839 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:42.587+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:42.839 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:42.587+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:42.839 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:42 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:42.587+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:43.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:42.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:43.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:42.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:43.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:42.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:43.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:42 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:42.892+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:43.740 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:43.584+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:43.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:43.584+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:43.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:43.584+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:43.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:43.584+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:43.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:43.489+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:43.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:43.489+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:43.825 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:43.489+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:43.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:43.681+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:43.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:43.681+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:43.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:43.681+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:43.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:43.681+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:43.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:43 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:43.681+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:44.240 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:43.810+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:44.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:43.810+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:44.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:43 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:43.810+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:44.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:43.861+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:44.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:43.861+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:44.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:43.861+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:44.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:43.861+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:44.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:43 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:43.861+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:44.722 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:44.443+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:54.791073+0000 front 2024-04-25T13:15:54.791116+0000 (oldest deadline 2024-04-25T13:16:19.991842+0000) 2024-04-25T13:16:44.763 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:44.443+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.493137+0000 front 2024-04-25T13:16:03.493050+0000 (oldest deadline 2024-04-25T13:16:28.793013+0000) 2024-04-25T13:16:44.763 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:44.443+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.694227+0000 front 2024-04-25T13:16:11.694159+0000 (oldest deadline 2024-04-25T13:16:34.594024+0000) 2024-04-25T13:16:44.763 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7[51250]: 2024-04-25T13:16:44.443+0000 7f5bbaf8d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:20.494928+0000 front 2024-04-25T13:16:20.495009+0000 (oldest deadline 2024-04-25T13:16:43.995063+0000) 2024-04-25T13:16:44.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:44.589+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:44.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:44.589+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:44.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:44.589+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:44.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:44.589+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:44.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:44.784+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:44.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:44.784+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:44.992 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:44.784+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:44.992 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:44 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:44.784+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:45.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:44.722+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:45.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:44.722+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:45.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:44.722+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:45.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:44.722+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:45.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:44 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:44.722+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:45.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:44.876+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:45.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:44.876+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:45.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:44.876+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:45.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:44.876+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:45.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:44 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:44.876+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:45.896 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:45 smithi104 podman[69332]: 2024-04-25 13:16:45.622107487 +0000 UTC m=+5.451607214 container died 19d2f4d2b815698a3dc1a130a02308611bb7a6cc9aa531885d392a557269acd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T13:16:45.896 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:45.675+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:45.896 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:45.675+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:45.896 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:45.675+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:45.896 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:45.675+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:45.896 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:45.675+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:45.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:45.745+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:45.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:45.745+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:45.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:45.745+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:45.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:45.745+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:45.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:45.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:45.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:45.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:45.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:45.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:45.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:45.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:45.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:45 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:45.581+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:46.271 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:45 smithi104 podman[69332]: 2024-04-25 13:16:45.987146569 +0000 UTC m=+5.816646292 container cleanup 19d2f4d2b815698a3dc1a130a02308611bb7a6cc9aa531885d392a557269acd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T13:16:46.271 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:45 smithi104 bash[69332]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7 2024-04-25T13:16:46.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:45.831+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:46.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:45.831+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:46.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:45.831+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:46.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:45.831+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:46.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:45 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:45.831+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:46.575 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:46 smithi104 podman[69349]: 2024-04-25 13:16:46.271564997 +0000 UTC m=+0.650850837 container remove 19d2f4d2b815698a3dc1a130a02308611bb7a6cc9aa531885d392a557269acd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:16:46.949 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:46.659+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:46.949 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:46.659+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:46.659+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:46.659+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:46 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:46.659+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:46 smithi104 podman[69442]: 2024-04-25 13:16:46.514334702 +0000 UTC m=+0.017692814 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:46 smithi104 podman[69442]: 2024-04-25 13:16:46.614099729 +0000 UTC m=+0.117457870 container create b0c0c92e79bbd3d52f482ba007bf95360b99399268b5865239a5a1f473d080a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:46 smithi104 podman[69442]: 2024-04-25 13:16:46.850802627 +0000 UTC m=+0.354160729 container init b0c0c92e79bbd3d52f482ba007bf95360b99399268b5865239a5a1f473d080a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T13:16:46.950 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:46 smithi104 podman[69442]: 2024-04-25 13:16:46.853959927 +0000 UTC m=+0.357318026 container start b0c0c92e79bbd3d52f482ba007bf95360b99399268b5865239a5a1f473d080a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:16:46.951 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:46 smithi104 podman[69442]: 2024-04-25 13:16:46.949910616 +0000 UTC m=+0.453268743 container attach b0c0c92e79bbd3d52f482ba007bf95360b99399268b5865239a5a1f473d080a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:16:46.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:46.767+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:46.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:46.767+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:46.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:46.767+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:46.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:46.767+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:46.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:46.567+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:46.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:46.567+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:46.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:46.567+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:46.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:46.567+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:46.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:46 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:46.567+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:47.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:46.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:47.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:46.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:47.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:46.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:47.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:46.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:47.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:46 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:46.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:47.325 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 25 13:16:47 smithi104 podman[69442]: 2024-04-25 13:16:47.1464522 +0000 UTC m=+0.649810300 container died b0c0c92e79bbd3d52f482ba007bf95360b99399268b5865239a5a1f473d080a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:16:47.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T13:16:47.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:47.871 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T13:16:47.871 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T13:16:47.871 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.8 2024-04-25T13:16:47.897 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:47.773+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:47.897 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:47.773+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:47.773+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:47.773+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:47.615+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:47.615+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:47.615+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:47.615+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:47.898 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:47 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:47.615+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:48.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:47.794+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:48.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:47.794+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:48.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:47.794+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:48.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:47.794+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:48.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:47 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:47.794+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:48.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:47.682+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:48.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:47.682+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:48.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:47.682+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:48.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:47.682+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:48.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:47 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:47.682+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:48.201 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:47 smithi178 systemd[1]: Stopping Ceph osd.8 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:48.490 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.201+0000 7fa8d9752700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:48.490 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.201+0000 7fa8d9752700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-25T13:16:48.491 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.201+0000 7fa8d9752700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:48.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.815+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:48.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.815+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:48.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.815+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:48.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:48.815+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:48.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:48.653+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:48.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:48.653+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:48.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:48.653+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:48.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:48.653+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:48.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:48 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:48.653+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:49.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:48.795+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:49.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:48.795+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:49.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:48.795+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:49.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:48.795+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:49.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:48 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:48.795+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:49.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:48.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:49.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:48.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:49.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:48.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:49.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:48.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:49.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:48 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:48.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:49.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:49.832+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:49.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:49.832+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:49.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:49.832+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:49.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:49.832+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:49.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:49.696+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:49.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:49.696+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:49.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:49.696+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:49.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:49.696+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:49.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:49.696+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:50.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:49 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:49.748+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:50.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:49 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:49.748+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:50.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:49 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:49.748+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:50.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:49 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:49.748+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:50.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:49 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:49.748+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:50.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:49.718+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:50.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:49.718+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:50.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:49.718+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:50.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:49.718+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:50.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:49 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:49.718+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:50.990 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:50.823+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:50.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:50.823+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:50.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:50.823+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:50.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:50.823+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:50.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:50.823+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:27.463996+0000 front 2024-04-25T13:16:27.464079+0000 (oldest deadline 2024-04-25T13:16:50.364056+0000) 2024-04-25T13:16:50.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:50.727+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:50.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:50.727+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:50.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:50.727+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:50.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:50.727+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:50.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:50 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:50.727+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:51.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:50.796+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:51.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:50.796+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:51.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:50.796+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:51.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:50.797+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:51.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:50 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:50.797+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:51.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:50.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:51.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:50.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:51.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:50.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:51.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:50.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:51.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:50 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:50.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:52.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:51.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:52.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:51.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:52.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:51.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:52.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:51.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:52.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:51 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:51.807+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:52.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:51.680+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:52.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:51.680+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:52.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:51.680+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:52.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:51.680+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:52.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:51.680+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:52.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:51 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:51.680+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:52.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:51.867+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:52.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:51.867+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:52.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:51.867+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:52.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:51.867+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:52.241 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:51.867+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:27.463996+0000 front 2024-04-25T13:16:27.464079+0000 (oldest deadline 2024-04-25T13:16:50.364056+0000) 2024-04-25T13:16:52.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:51.751+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:52.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:51.751+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:52.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:51.751+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:52.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:51.751+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:52.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:51 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:51.751+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:52.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:52.709+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:52.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:52.709+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:52.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:52.709+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:52.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:52.709+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:52.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:52.709+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:52.991 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:52.855+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:53.758221+0000 front 2024-04-25T13:15:53.758174+0000 (oldest deadline 2024-04-25T13:16:19.058109+0000) 2024-04-25T13:16:52.992 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:52.855+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:04.859551+0000 front 2024-04-25T13:16:04.859742+0000 (oldest deadline 2024-04-25T13:16:29.460392+0000) 2024-04-25T13:16:52.992 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:52.855+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:12.262041+0000 front 2024-04-25T13:16:12.261816+0000 (oldest deadline 2024-04-25T13:16:37.561810+0000) 2024-04-25T13:16:52.992 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:52.855+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:18.063101+0000 front 2024-04-25T13:16:18.063114+0000 (oldest deadline 2024-04-25T13:16:43.962951+0000) 2024-04-25T13:16:52.992 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:52 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8[52210]: 2024-04-25T13:16:52.855+0000 7fa8d1d3e700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:27.463996+0000 front 2024-04-25T13:16:27.464079+0000 (oldest deadline 2024-04-25T13:16:50.364056+0000) 2024-04-25T13:16:53.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:52.773+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:53.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:52.773+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:53.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:52.773+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:53.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:52.773+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:53.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:52 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:52.773+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:53.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:52.641+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:53.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:52.641+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:53.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:52.641+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:53.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:52.641+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:53.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:52.641+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:53.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:52 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:52.641+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:53.851 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:53 smithi178 podman[70410]: 2024-04-25 13:16:53.571627754 +0000 UTC m=+5.503032992 container died 0e97cc17e8adfb50304d970c9271468714c20754cea99783842ad52d8416af35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:16:53.852 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:53.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:53.852 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:53.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:53.852 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:53.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:53.852 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:53.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:53.852 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:53.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:54.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:53.793+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:54.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:53.793+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:54.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:53.793+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:54.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:53.793+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:54.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:53.793+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:54.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:53 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:53.793+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:16:54.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:53.612+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:54.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:53.612+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:54.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:53.612+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:54.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:53.612+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:54.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:53.612+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:54.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:53 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:53.612+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:54.219 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:53 smithi178 podman[70410]: 2024-04-25 13:16:53.951697844 +0000 UTC m=+5.883103082 container cleanup 0e97cc17e8adfb50304d970c9271468714c20754cea99783842ad52d8416af35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T13:16:54.219 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:53 smithi178 bash[70410]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8 2024-04-25T13:16:54.490 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:54 smithi178 podman[70427]: 2024-04-25 13:16:54.219765342 +0000 UTC m=+0.650992507 container remove 0e97cc17e8adfb50304d970c9271468714c20754cea99783842ad52d8416af35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T13:16:54.822 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:54 smithi178 podman[70517]: 2024-04-25 13:16:54.472295466 +0000 UTC m=+0.018396334 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:54 smithi178 podman[70517]: 2024-04-25 13:16:54.587363495 +0000 UTC m=+0.133464429 container create 9888fed95e53b9632be7a300c0a63cc829f229ff86dd2030bb77ac67e3b8ce4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:54.693+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:54.693+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:54.693+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:54.693+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:54.693+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:54.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:54 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:54.693+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:16:55.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:54.785+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:55.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:54.785+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:55.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:54.785+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:55.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:54.785+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:55.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:54.785+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:55.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:54 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:54.785+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:16:55.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:54.643+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:55.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:54.643+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:55.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:54.643+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:55.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:54.643+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:55.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:54.643+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:55.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:54 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:54.643+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:55.125 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:54 smithi178 podman[70517]: 2024-04-25 13:16:54.823296883 +0000 UTC m=+0.369397752 container init 9888fed95e53b9632be7a300c0a63cc829f229ff86dd2030bb77ac67e3b8ce4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T13:16:55.125 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:54 smithi178 podman[70517]: 2024-04-25 13:16:54.826900819 +0000 UTC m=+0.373001697 container start 9888fed95e53b9632be7a300c0a63cc829f229ff86dd2030bb77ac67e3b8ce4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:16:55.125 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:54 smithi178 podman[70517]: 2024-04-25 13:16:54.898396492 +0000 UTC m=+0.444497370 container attach 9888fed95e53b9632be7a300c0a63cc829f229ff86dd2030bb77ac67e3b8ce4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:16:55.400 INFO:journalctl@ceph.osd.8.smithi178.stdout:Apr 25 13:16:55 smithi178 podman[70517]: 2024-04-25 13:16:55.125396102 +0000 UTC m=+0.671496970 container died 9888fed95e53b9632be7a300c0a63cc829f229ff86dd2030bb77ac67e3b8ce4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-25T13:16:55.804 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T13:16:55.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:16:55.881 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T13:16:55.881 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T13:16:55.882 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.9 2024-04-25T13:16:55.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:55 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:55.676+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:55.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:55 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:55.676+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:55.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:55 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:55.676+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:55.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:55 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:55.676+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:55.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:55 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:55.676+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:55.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:55 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:55.676+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:16:56.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:55.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:56.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:55.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:56.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:55.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:56.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:55.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:56.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:55.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:56.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:55 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:55.671+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:55.825+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:55.825+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:55.825+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:55.825+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:55.825+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:55.825+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:16:56.206 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:55 smithi038 systemd[1]: Stopping Ceph osd.9 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:16:56.207 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.205+0000 7f4471972700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:56.207 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.205+0000 7f4471972700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-25T13:16:56.207 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.205+0000 7f4471972700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:16:56.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:56 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:56.672+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:56.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:56 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:56.672+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:56.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:56 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:56.672+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:56.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:56 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:56.672+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:56.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:56 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:56.672+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:56.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:56 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:56.672+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:16:57.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.803+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:57.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.803+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:57.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.803+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:57.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.803+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:57.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.803+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:57.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:56 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:56.803+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:16:57.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:56 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:56.661+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:57.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:56 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:56.661+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:57.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:56 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:56.661+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:57.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:56 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:56.661+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:57.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:56 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:56.661+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:57.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:56 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:56.661+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:57.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:57.687+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:57.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:57.687+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:57.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:57.687+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:57.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:57.687+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:57.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:57.687+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:57.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:57.687+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:16:58.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:57 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:57.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:58.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:57 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:57.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:58.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:57 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:57.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:58.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:57 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:57.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:58.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:57 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:57.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:58.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:57 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:57.684+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:58.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:57 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:57.842+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:58.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:57 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:57.842+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:58.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:57 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:57.842+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:58.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:57 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:57.842+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:58.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:57 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:57.842+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:58.323 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:57 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:57.842+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:16:58.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:16:58.825 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:16:58.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:16:58.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:16:58.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:16:58.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:16:58.826 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:58 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:58.669+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:16:58.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:58.649+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:58.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:58.649+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:58.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:58.649+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:58.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:58.649+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:58.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:58.649+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:58.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:58 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:58.649+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:16:59.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:16:59.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:16:59.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:16:59.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:16:59.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:16:59.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:16:59.074 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:58 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:58.792+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.526388+0000 front 2024-04-25T13:16:33.526356+0000 (oldest deadline 2024-04-25T13:16:58.226501+0000) 2024-04-25T13:16:59.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:59 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:59.644+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:16:59.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:59 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:59.644+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:16:59.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:59 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:59.644+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:16:59.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:59 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:59.644+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:16:59.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:59 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:59.644+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:16:59.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:16:59 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:16:59.644+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:17:00.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:16:59 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:16:59.776+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.526388+0000 front 2024-04-25T13:16:33.526356+0000 (oldest deadline 2024-04-25T13:16:58.226501+0000) 2024-04-25T13:17:00.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:00.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:00.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:00.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:00.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:00.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:00.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:16:59 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:16:59.696+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:00.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:00.650+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:00.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:00.650+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:00.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:00.650+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:00.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:00.650+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:00.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:00.650+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:00.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:00 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:00.650+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:01.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:47.116986+0000 front 2024-04-25T13:15:47.117093+0000 (oldest deadline 2024-04-25T13:16:11.817196+0000) 2024-04-25T13:17:01.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:55.219038+0000 front 2024-04-25T13:15:55.219019+0000 (oldest deadline 2024-04-25T13:16:19.319015+0000) 2024-04-25T13:17:01.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:05.120742+0000 front 2024-04-25T13:16:05.120628+0000 (oldest deadline 2024-04-25T13:16:26.220738+0000) 2024-04-25T13:17:01.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.322865+0000 front 2024-04-25T13:16:11.322826+0000 (oldest deadline 2024-04-25T13:16:35.422959+0000) 2024-04-25T13:17:01.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:19.524654+0000 front 2024-04-25T13:16:19.524570+0000 (oldest deadline 2024-04-25T13:16:43.024266+0000) 2024-04-25T13:17:01.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.426115+0000 front 2024-04-25T13:16:29.426019+0000 (oldest deadline 2024-04-25T13:16:53.525949+0000) 2024-04-25T13:17:01.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:00 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9[61585]: 2024-04-25T13:17:00.804+0000 7f4469f5e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.526388+0000 front 2024-04-25T13:16:33.526356+0000 (oldest deadline 2024-04-25T13:16:58.226501+0000) 2024-04-25T13:17:01.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:01.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:01.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:01.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:01.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:01.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:01.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:00 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:00.662+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:01.805 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:01 smithi038 podman[85395]: 2024-04-25 13:17:01.541381649 +0000 UTC m=+5.455564996 container died b470e41bdd207c79f17de96e3d5cfc150e3537b3148a8d2ec99800587b1fdd0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T13:17:01.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:01.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:01.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:01.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:01.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:01.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:01.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:01.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:01.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:01.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:01.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:01.683+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:02.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:01 smithi038 podman[85395]: 2024-04-25 13:17:01.90480233 +0000 UTC m=+5.818985685 container cleanup b470e41bdd207c79f17de96e3d5cfc150e3537b3148a8d2ec99800587b1fdd0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:17:02.073 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:01 smithi038 bash[85395]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9 2024-04-25T13:17:02.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:02.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:02.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:02.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:02.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:02.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:02.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:01 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:01.687+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:02.531 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:02 smithi038 podman[85412]: 2024-04-25 13:17:02.181382032 +0000 UTC m=+0.634590299 container remove b470e41bdd207c79f17de96e3d5cfc150e3537b3148a8d2ec99800587b1fdd0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T13:17:02.784 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:02 smithi038 podman[85501]: 2024-04-25 13:17:02.532046276 +0000 UTC m=+0.113619681 container create 8b983f156c2a2049f8e44c2dcdf1e3fe0e680c2dbd888bada4afcc67eb26fd66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T13:17:02.784 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:02 smithi038 podman[85501]: 2024-04-25 13:17:02.437821565 +0000 UTC m=+0.019394984 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:02.784 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:02 smithi038 podman[85501]: 2024-04-25 13:17:02.701125314 +0000 UTC m=+0.282698718 container init 8b983f156c2a2049f8e44c2dcdf1e3fe0e680c2dbd888bada4afcc67eb26fd66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:17:02.784 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:02 smithi038 podman[85501]: 2024-04-25 13:17:02.704308131 +0000 UTC m=+0.285881538 container start 8b983f156c2a2049f8e44c2dcdf1e3fe0e680c2dbd888bada4afcc67eb26fd66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:17:02.785 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:02 smithi038 podman[85501]: 2024-04-25 13:17:02.784335905 +0000 UTC m=+0.365909326 container attach 8b983f156c2a2049f8e44c2dcdf1e3fe0e680c2dbd888bada4afcc67eb26fd66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:17:02.994 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:02 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:02.730+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:02.994 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:02 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:02.730+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:02.994 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:02 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:02.730+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:02.994 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:02 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:02.730+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:02.995 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:02 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:02.730+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:02.995 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:02 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:02.730+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:03.072 INFO:journalctl@ceph.osd.9.smithi038.stdout:Apr 25 13:17:03 smithi038 podman[85501]: 2024-04-25 13:17:03.010288707 +0000 UTC m=+0.591862121 container died 8b983f156c2a2049f8e44c2dcdf1e3fe0e680c2dbd888bada4afcc67eb26fd66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:17:03.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:03.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:03.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:03.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:03.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:03.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:03.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:02 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:02.683+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:03.579 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T13:17:03.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:03.618 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T13:17:03.618 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T13:17:03.619 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.10 2024-04-25T13:17:03.940 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 systemd[1]: Stopping Ceph osd.10 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:03.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.713+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:03.942 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.939+0000 7f169bbf3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:03.942 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.939+0000 7f169bbf3700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-25T13:17:03.942 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:03 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:03.939+0000 7f169bbf3700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:17:03.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:03 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:03.689+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:03.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:03 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:03.689+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:03.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:03 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:03.689+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:03.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:03 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:03.689+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:03.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:03 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:03.689+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:03.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:03 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:03.689+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:04.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:04 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:04.736+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:04.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:04 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:04.736+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:04.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:04 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:04.736+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:04.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:04 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:04.736+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:04.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:04 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:04.736+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:04.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:04 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:04.736+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:05.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:05.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:05.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:05.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:05.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:05.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:05.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:04 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:04.701+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:06.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:06.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:06.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:06.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:06.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:06.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:06.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:05 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:05.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:06.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:05 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:05.750+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:06.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:05 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:05.750+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:06.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:05 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:05.750+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:06.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:05 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:05.750+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:06.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:05 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:05.750+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:06.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:05 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:05.750+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:07.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:07.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:07.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:07.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:07.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:07.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:07.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:06 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:06.640+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:07.240 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:07.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:07.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:07.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:07.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:07.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:07.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:06 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:06.775+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:07.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:07.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:07.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:07.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:07.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:07.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:07.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:07 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:07.726+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:08.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:08.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:08.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:08.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:08.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:08.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:08.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:07 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:07.676+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:08.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:08.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:08.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:08.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:08.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:08.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:08.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:08 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:08.733+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:09.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.289639+0000 front 2024-04-25T13:15:46.289626+0000 (oldest deadline 2024-04-25T13:16:10.389631+0000) 2024-04-25T13:17:09.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:56.290575+0000 front 2024-04-25T13:15:56.290628+0000 (oldest deadline 2024-04-25T13:16:18.590710+0000) 2024-04-25T13:17:09.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:01.992519+0000 front 2024-04-25T13:16:01.992645+0000 (oldest deadline 2024-04-25T13:16:26.692359+0000) 2024-04-25T13:17:09.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:09.994158+0000 front 2024-04-25T13:16:09.994333+0000 (oldest deadline 2024-04-25T13:16:34.694158+0000) 2024-04-25T13:17:09.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:16.995032+0000 front 2024-04-25T13:16:16.994992+0000 (oldest deadline 2024-04-25T13:16:42.895158+0000) 2024-04-25T13:17:09.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:25.696782+0000 front 2024-04-25T13:16:25.696672+0000 (oldest deadline 2024-04-25T13:16:50.996493+0000) 2024-04-25T13:17:09.076 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:08 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10[58042]: 2024-04-25T13:17:08.654+0000 7f16981e7700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-04-25T13:16:33.897767+0000 front 2024-04-25T13:16:33.897852+0000 (oldest deadline 2024-04-25T13:16:57.997588+0000) 2024-04-25T13:17:09.575 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:09 smithi104 podman[69559]: 2024-04-25 13:17:09.3253338 +0000 UTC m=+5.514373560 container died d4b5d15b31243eb6c3883ffdc7c1c6737e46bce766bb221577b5791b8397526c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T13:17:10.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:09 smithi104 podman[69559]: 2024-04-25 13:17:09.639695914 +0000 UTC m=+5.828735659 container cleanup d4b5d15b31243eb6c3883ffdc7c1c6737e46bce766bb221577b5791b8397526c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T13:17:10.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:09 smithi104 bash[69559]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10 2024-04-25T13:17:10.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:09 smithi104 podman[69576]: 2024-04-25 13:17:09.832285124 +0000 UTC m=+0.504431439 container remove d4b5d15b31243eb6c3883ffdc7c1c6737e46bce766bb221577b5791b8397526c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:17:10.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:10.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:10.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:10.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:10.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:10.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:10.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:09 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:09.784+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:10.575 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:10 smithi104 podman[69662]: 2024-04-25 13:17:10.156065758 +0000 UTC m=+0.018046884 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:10.575 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:10 smithi104 podman[69662]: 2024-04-25 13:17:10.266404433 +0000 UTC m=+0.128385555 container create df385df33708a049dad05da974f049d83b95d6d8b255769cbc461fbbe0a81202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:17:10.575 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:10 smithi104 podman[69662]: 2024-04-25 13:17:10.494076343 +0000 UTC m=+0.356057460 container init df385df33708a049dad05da974f049d83b95d6d8b255769cbc461fbbe0a81202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:17:10.576 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:10 smithi104 podman[69662]: 2024-04-25 13:17:10.497120469 +0000 UTC m=+0.359101597 container start df385df33708a049dad05da974f049d83b95d6d8b255769cbc461fbbe0a81202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T13:17:11.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:10 smithi104 podman[69662]: 2024-04-25 13:17:10.577221066 +0000 UTC m=+0.439202211 container attach df385df33708a049dad05da974f049d83b95d6d8b255769cbc461fbbe0a81202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:17:11.075 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:10 smithi104 podman[69662]: 2024-04-25 13:17:10.787662496 +0000 UTC m=+0.649643607 container died df385df33708a049dad05da974f049d83b95d6d8b255769cbc461fbbe0a81202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T13:17:11.240 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:11.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:11.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:11.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:11.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:11.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:11.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:10 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:10.803+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:11.385 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T13:17:11.407 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:11 smithi104 podman[69688]: 2024-04-25 13:17:11.355595587 +0000 UTC m=+0.561685855 container remove df385df33708a049dad05da974f049d83b95d6d8b255769cbc461fbbe0a81202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T13:17:11.408 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:11 smithi104 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-25T13:17:11.408 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:11 smithi104 systemd[1]: Stopped Ceph osd.10 for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:17:11.408 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 25 13:17:11 smithi104 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.10.service: Consumed 9.757s CPU time. 2024-04-25T13:17:11.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:11.423 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T13:17:11.424 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T13:17:11.424 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.11 2024-04-25T13:17:11.705 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 systemd[1]: Stopping Ceph osd.11 for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:11.990 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.704+0000 7f2793a96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.704+0000 7f2793a96700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-25T13:17:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.704+0000 7f2793a96700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:17:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:11.991 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:11.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:11.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:11.992 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:11 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:11.823+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:13.240 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:13.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:13.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:13.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:13.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:13.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:13.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:12 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:12.832+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:14.240 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:14.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:14.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:14.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:14.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:14.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:14.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:13 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:13.792+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:15.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:15.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:15.243 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:15.243 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:15.243 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:15.243 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:15.243 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:15.243 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:14 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:14.785+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-04-25T13:16:49.591320+0000 front 2024-04-25T13:16:49.591293+0000 (oldest deadline 2024-04-25T13:17:14.291277+0000) 2024-04-25T13:17:16.240 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-04-25T13:15:46.581700+0000 front 2024-04-25T13:15:46.581652+0000 (oldest deadline 2024-04-25T13:16:11.282002+0000) 2024-04-25T13:17:16.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-25T13:15:57.183325+0000 front 2024-04-25T13:15:57.183384+0000 (oldest deadline 2024-04-25T13:16:23.083296+0000) 2024-04-25T13:17:16.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-04-25T13:16:03.083637+0000 front 2024-04-25T13:16:03.083665+0000 (oldest deadline 2024-04-25T13:16:25.983859+0000) 2024-04-25T13:17:16.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-04-25T13:16:11.785758+0000 front 2024-04-25T13:16:11.785808+0000 (oldest deadline 2024-04-25T13:16:34.685662+0000) 2024-04-25T13:17:16.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-25T13:16:21.087144+0000 front 2024-04-25T13:16:21.087242+0000 (oldest deadline 2024-04-25T13:16:45.087378+0000) 2024-04-25T13:17:16.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-04-25T13:16:29.188543+0000 front 2024-04-25T13:16:29.188596+0000 (oldest deadline 2024-04-25T13:16:53.888580+0000) 2024-04-25T13:17:16.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-25T13:16:42.590547+0000 front 2024-04-25T13:16:42.590665+0000 (oldest deadline 2024-04-25T13:17:06.690394+0000) 2024-04-25T13:17:16.242 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:15 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11[58850]: 2024-04-25T13:17:15.802+0000 7f279008a700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-04-25T13:16:49.591320+0000 front 2024-04-25T13:16:49.591293+0000 (oldest deadline 2024-04-25T13:17:14.291277+0000) 2024-04-25T13:17:17.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:17 smithi178 podman[70633]: 2024-04-25 13:17:17.030111034 +0000 UTC m=+5.413473053 container died 34c1b2ca4a3ad8eedd3073995db08f48290ad42a11a140006b62b984e4fe4c90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:17:17.557 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:17 smithi178 podman[70633]: 2024-04-25 13:17:17.364844036 +0000 UTC m=+5.748206049 container cleanup 34c1b2ca4a3ad8eedd3073995db08f48290ad42a11a140006b62b984e4fe4c90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T13:17:17.557 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:17 smithi178 bash[70633]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11 2024-04-25T13:17:17.974 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:17 smithi178 podman[70649]: 2024-04-25 13:17:17.557612401 +0000 UTC m=+0.525584963 container remove 34c1b2ca4a3ad8eedd3073995db08f48290ad42a11a140006b62b984e4fe4c90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:17:18.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:17 smithi178 podman[70741]: 2024-04-25 13:17:17.877615041 +0000 UTC m=+0.018828966 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:18.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:17 smithi178 podman[70741]: 2024-04-25 13:17:17.974943178 +0000 UTC m=+0.116157079 container create 22bc7a125a09e3decf9291f6b9682db3bfdf4211818636c3f2635b4d5236669b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T13:17:18.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:18 smithi178 podman[70741]: 2024-04-25 13:17:18.177333387 +0000 UTC m=+0.318547288 container init 22bc7a125a09e3decf9291f6b9682db3bfdf4211818636c3f2635b4d5236669b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:17:18.241 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:18 smithi178 podman[70741]: 2024-04-25 13:17:18.18028833 +0000 UTC m=+0.321502240 container start 22bc7a125a09e3decf9291f6b9682db3bfdf4211818636c3f2635b4d5236669b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:17:18.740 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:18 smithi178 podman[70741]: 2024-04-25 13:17:18.264707638 +0000 UTC m=+0.405921551 container attach 22bc7a125a09e3decf9291f6b9682db3bfdf4211818636c3f2635b4d5236669b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T13:17:18.741 INFO:journalctl@ceph.osd.11.smithi178.stdout:Apr 25 13:17:18 smithi178 podman[70741]: 2024-04-25 13:17:18.471428081 +0000 UTC m=+0.612641996 container died 22bc7a125a09e3decf9291f6b9682db3bfdf4211818636c3f2635b4d5236669b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:17:19.153 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T13:17:19.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:19.191 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T13:17:19.191 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T13:17:19.191 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.a 2024-04-25T13:17:19.506 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:17:19 smithi038 systemd[1]: Stopping Ceph mds.a for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:19.822 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:17:19 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a[68455]: 2024-04-25T13:17:19.505+0000 7ffa91d73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:19.823 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:17:19 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a[68455]: 2024-04-25T13:17:19.505+0000 7ffa91d73700 -1 mds.a *** got signal Terminated *** 2024-04-25T13:17:20.800 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:17:20 smithi038 podman[85608]: 2024-04-25 13:17:20.726578234 +0000 UTC m=+1.330383856 container died a7e2e7b9f3baf1c32d754c315a48088ec45066c03e8f29f2496c0418cd169a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:17:21.366 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:17:21 smithi038 podman[85608]: 2024-04-25 13:17:21.089601673 +0000 UTC m=+1.693407287 container cleanup a7e2e7b9f3baf1c32d754c315a48088ec45066c03e8f29f2496c0418cd169a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T13:17:21.366 INFO:journalctl@ceph.mds.a.smithi038.stdout:Apr 25 13:17:21 smithi038 bash[85608]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-a 2024-04-25T13:17:21.537 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T13:17:21.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:21.574 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T13:17:21.575 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T13:17:21.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.d 2024-04-25T13:17:21.919 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:17:21 smithi038 systemd[1]: Stopping Ceph mds.d for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:22.322 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:17:21 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d[67493]: 2024-04-25T13:17:21.918+0000 7f4ce9b35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:22.323 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:17:21 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d[67493]: 2024-04-25T13:17:21.918+0000 7f4ce9b35700 -1 mds.d *** got signal Terminated *** 2024-04-25T13:17:26.212 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:17:25 smithi038 podman[85739]: 2024-04-25 13:17:25.925040617 +0000 UTC m=+4.123038477 container died 40c4b9555870d8c1a639c6c0e1e763e081896df0d242af0642a6915847016e77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:17:26.573 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:17:26 smithi038 podman[85739]: 2024-04-25 13:17:26.410286231 +0000 UTC m=+4.608284093 container cleanup 40c4b9555870d8c1a639c6c0e1e763e081896df0d242af0642a6915847016e77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T13:17:26.573 INFO:journalctl@ceph.mds.d.smithi038.stdout:Apr 25 13:17:26 smithi038 bash[85739]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-d 2024-04-25T13:17:26.844 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T13:17:26.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:26.880 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T13:17:26.880 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T13:17:26.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.g 2024-04-25T13:17:27.198 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:17:26 smithi038 systemd[1]: Stopping Ceph mds.g for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:27.572 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:17:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g[68104]: 2024-04-25T13:17:27.196+0000 7f6cdcd7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:27.573 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:17:27 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g[68104]: 2024-04-25T13:17:27.196+0000 7f6cdcd7b700 -1 mds.g *** got signal Terminated *** 2024-04-25T13:17:28.981 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:17:28 smithi038 podman[85859]: 2024-04-25 13:17:28.728641916 +0000 UTC m=+1.623716284 container died 3bb9cf690114e204729ed56088d2d4dd9e17347d81c80eebc712d270a730e930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T13:17:29.322 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:17:29 smithi038 podman[85859]: 2024-04-25 13:17:29.081266686 +0000 UTC m=+1.976341064 container cleanup 3bb9cf690114e204729ed56088d2d4dd9e17347d81c80eebc712d270a730e930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T13:17:29.323 INFO:journalctl@ceph.mds.g.smithi038.stdout:Apr 25 13:17:29 smithi038 bash[85859]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-g 2024-04-25T13:17:29.523 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T13:17:29.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:29.555 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T13:17:29.555 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T13:17:29.555 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.j 2024-04-25T13:17:29.884 INFO:journalctl@ceph.mds.j.smithi038.stdout:Apr 25 13:17:29 smithi038 systemd[1]: Stopping Ceph mds.j for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:30.323 INFO:journalctl@ceph.mds.j.smithi038.stdout:Apr 25 13:17:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-j[67140]: 2024-04-25T13:17:29.884+0000 7f9bdd78d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:30.323 INFO:journalctl@ceph.mds.j.smithi038.stdout:Apr 25 13:17:29 smithi038 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-j[67140]: 2024-04-25T13:17:29.884+0000 7f9bdd78d700 -1 mds.j *** got signal Terminated *** 2024-04-25T13:17:31.322 INFO:journalctl@ceph.mds.j.smithi038.stdout:Apr 25 13:17:31 smithi038 podman[85980]: 2024-04-25 13:17:31.066448007 +0000 UTC m=+1.284090093 container died 6fe915df10dde141bdd90806902b7d0eaab9a32cb8bdc96c091c5ff4fd2dec11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:17:31.690 INFO:journalctl@ceph.mds.j.smithi038.stdout:Apr 25 13:17:31 smithi038 podman[85980]: 2024-04-25 13:17:31.537690502 +0000 UTC m=+1.755332593 container cleanup 6fe915df10dde141bdd90806902b7d0eaab9a32cb8bdc96c091c5ff4fd2dec11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T13:17:31.690 INFO:journalctl@ceph.mds.j.smithi038.stdout:Apr 25 13:17:31 smithi038 bash[85980]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-j 2024-04-25T13:17:31.891 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T13:17:31.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:31.923 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T13:17:31.923 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T13:17:31.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.b 2024-04-25T13:17:32.213 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:17:31 smithi104 systemd[1]: Stopping Ceph mds.b for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:32.471 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:17:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: 2024-04-25T13:17:32.212+0000 7fa792eef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:32.471 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:17:32 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b[63269]: 2024-04-25T13:17:32.212+0000 7fa792eef700 -1 mds.b *** got signal Terminated *** 2024-04-25T13:17:36.513 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:17:36 smithi104 podman[69769]: 2024-04-25 13:17:36.227204366 +0000 UTC m=+4.112482270 container died 8270cd36762eb1769a3cdf3653fdd988b48c595fe18bdb66538755e08a42b31f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:17:36.825 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:17:36 smithi104 podman[69769]: 2024-04-25 13:17:36.612812256 +0000 UTC m=+4.498090164 container cleanup 8270cd36762eb1769a3cdf3653fdd988b48c595fe18bdb66538755e08a42b31f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:17:36.825 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 25 13:17:36 smithi104 bash[69769]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-b 2024-04-25T13:17:36.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T13:17:37.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:37.029 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T13:17:37.030 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T13:17:37.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.e 2024-04-25T13:17:37.575 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:37 smithi104 systemd[1]: Stopping Ceph mds.e for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:37.575 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e[62650]: 2024-04-25T13:17:37.341+0000 7faf1bcac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:37.575 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:37 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e[62650]: 2024-04-25T13:17:37.341+0000 7faf1bcac700 -1 mds.e *** got signal Terminated *** 2024-04-25T13:17:41.026 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:40 smithi104 podman[69904]: 2024-04-25 13:17:40.753967296 +0000 UTC m=+3.511399392 container died e1d286e0388c506b886acb1e0ecb252f9cab15ca360d3b5bc04a7ec61082b4eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:17:41.317 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:41 smithi104 podman[69904]: 2024-04-25 13:17:41.125047001 +0000 UTC m=+3.882479098 container cleanup e1d286e0388c506b886acb1e0ecb252f9cab15ca360d3b5bc04a7ec61082b4eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:17:41.318 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:41 smithi104 bash[69904]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e 2024-04-25T13:17:41.549 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T13:17:41.569 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:41 smithi104 podman[69918]: 2024-04-25 13:17:41.317669283 +0000 UTC m=+0.559692173 container remove e1d286e0388c506b886acb1e0ecb252f9cab15ca360d3b5bc04a7ec61082b4eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T13:17:41.569 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:41 smithi104 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-25T13:17:41.570 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:41 smithi104 systemd[1]: Stopped Ceph mds.e for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:17:41.570 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 25 13:17:41 smithi104 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.e.service: Consumed 2.146s CPU time. 2024-04-25T13:17:41.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:41.581 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T13:17:41.582 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T13:17:41.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.h 2024-04-25T13:17:41.825 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:17:41 smithi104 systemd[1]: Stopping Ceph mds.h for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:42.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:17:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h[62957]: 2024-04-25T13:17:41.919+0000 7fd4b3753700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:42.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:17:41 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h[62957]: 2024-04-25T13:17:41.919+0000 7fd4b3753700 -1 mds.h *** got signal Terminated *** 2024-04-25T13:17:45.012 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:17:44 smithi104 podman[70026]: 2024-04-25 13:17:44.733842268 +0000 UTC m=+2.932524461 container died aa0d6bbbcaf99649d5c072487eb12815c4411b5e8542c0712371a0d9ae3f030b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:17:45.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:17:45 smithi104 podman[70026]: 2024-04-25 13:17:45.111860588 +0000 UTC m=+3.310542792 container cleanup aa0d6bbbcaf99649d5c072487eb12815c4411b5e8542c0712371a0d9ae3f030b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T13:17:45.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 25 13:17:45 smithi104 bash[70026]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-h 2024-04-25T13:17:45.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T13:17:45.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:45.572 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T13:17:45.572 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T13:17:45.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.k 2024-04-25T13:17:45.825 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:45 smithi104 systemd[1]: Stopping Ceph mds.k for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:46.325 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k[62322]: 2024-04-25T13:17:45.906+0000 7f7851b49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:46.325 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:45 smithi104 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k[62322]: 2024-04-25T13:17:45.906+0000 7f7851b49700 -1 mds.k *** got signal Terminated *** 2024-04-25T13:17:48.990 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:48 smithi104 podman[70147]: 2024-04-25 13:17:48.728527483 +0000 UTC m=+2.940151527 container died 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T13:17:49.257 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:49 smithi104 podman[70147]: 2024-04-25 13:17:49.074109835 +0000 UTC m=+3.285733871 container cleanup 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:17:49.257 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:49 smithi104 bash[70147]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k 2024-04-25T13:17:49.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T13:17:49.521 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:49 smithi104 podman[70161]: 2024-04-25 13:17:49.257892113 +0000 UTC m=+0.524852913 container remove 7ea9de500687c0db52ee87cddbcb24cf7e3d9a3af84a1792c09f44249a310793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:17:49.521 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:49 smithi104 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T13:17:49.521 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:49 smithi104 systemd[1]: Stopped Ceph mds.k for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:17:49.521 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 25 13:17:49 smithi104 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.k.service: Consumed 2.196s CPU time. 2024-04-25T13:17:49.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:49.531 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T13:17:49.531 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T13:17:49.532 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.c 2024-04-25T13:17:49.812 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:17:49 smithi178 systemd[1]: Stopping Ceph mds.c for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:50.072 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:17:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c[64007]: 2024-04-25T13:17:49.811+0000 7f63b197a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:50.072 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:17:49 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c[64007]: 2024-04-25T13:17:49.811+0000 7f63b197a700 -1 mds.c *** got signal Terminated *** 2024-04-25T13:17:52.991 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:17:52 smithi178 podman[70846]: 2024-04-25 13:17:52.733226137 +0000 UTC m=+3.012235649 container died 13ba5e7f6fdce18eb7cf0c9759d8b7c7a76579575e652e4891ce43a7562acb97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T13:17:53.353 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:17:53 smithi178 podman[70846]: 2024-04-25 13:17:53.103039227 +0000 UTC m=+3.382048740 container cleanup 13ba5e7f6fdce18eb7cf0c9759d8b7c7a76579575e652e4891ce43a7562acb97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:17:53.353 INFO:journalctl@ceph.mds.c.smithi178.stdout:Apr 25 13:17:53 smithi178 bash[70846]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-c 2024-04-25T13:17:53.484 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T13:17:53.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:53.518 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T13:17:53.518 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T13:17:53.518 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.f 2024-04-25T13:17:53.740 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:53 smithi178 systemd[1]: Stopping Ceph mds.f for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:54.240 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f[63694]: 2024-04-25T13:17:53.839+0000 7fbe5993f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:54.241 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:53 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f[63694]: 2024-04-25T13:17:53.839+0000 7fbe5993f700 -1 mds.f *** got signal Terminated *** 2024-04-25T13:17:56.990 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:56 smithi178 podman[70980]: 2024-04-25 13:17:56.73693564 +0000 UTC m=+3.002597085 container died 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:17:57.240 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:57 smithi178 podman[70980]: 2024-04-25 13:17:57.072676465 +0000 UTC m=+3.338337913 container cleanup 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:17:57.241 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:57 smithi178 bash[70980]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f 2024-04-25T13:17:57.489 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T13:17:57.509 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:57 smithi178 podman[70994]: 2024-04-25 13:17:57.249081294 +0000 UTC m=+0.512762895 container remove 89bffe7e0e368705cac610442dbed8b8b04052887a3e56562861fff5d315f7fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:17:57.509 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:57 smithi178 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-25T13:17:57.509 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:57 smithi178 systemd[1]: Stopped Ceph mds.f for d3627c2a-0302-11ef-bc93-c7b262605968. 2024-04-25T13:17:57.509 INFO:journalctl@ceph.mds.f.smithi178.stdout:Apr 25 13:17:57 smithi178 systemd[1]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.f.service: Consumed 2.272s CPU time. 2024-04-25T13:17:57.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:17:57.521 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T13:17:57.521 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T13:17:57.522 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.i 2024-04-25T13:17:57.826 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:17:57 smithi178 systemd[1]: Stopping Ceph mds.i for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:17:58.240 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:17:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i[64321]: 2024-04-25T13:17:57.826+0000 7f699f71d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:17:58.241 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:17:57 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i[64321]: 2024-04-25T13:17:57.826+0000 7f699f71d700 -1 mds.i *** got signal Terminated *** 2024-04-25T13:18:00.990 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:18:00 smithi178 podman[71102]: 2024-04-25 13:18:00.745836944 +0000 UTC m=+3.012119970 container died 2a632e5a93d5db05b72a6ce9afb0921f76dacf45c625dac3e0b7fa63f26c4377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-25T13:18:01.376 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:18:01 smithi178 podman[71102]: 2024-04-25 13:18:01.10964498 +0000 UTC m=+3.375928006 container cleanup 2a632e5a93d5db05b72a6ce9afb0921f76dacf45c625dac3e0b7fa63f26c4377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:18:01.377 INFO:journalctl@ceph.mds.i.smithi178.stdout:Apr 25 13:18:01 smithi178 bash[71102]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-i 2024-04-25T13:18:01.524 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T13:18:01.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:18:01.555 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T13:18:01.555 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T13:18:01.556 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.l 2024-04-25T13:18:01.990 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:01 smithi178 systemd[1]: Stopping Ceph mds.l for d3627c2a-0302-11ef-bc93-c7b262605968... 2024-04-25T13:18:01.991 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l[63372]: 2024-04-25T13:18:01.879+0000 7f00b96bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:18:01.991 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:01 smithi178 ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l[63372]: 2024-04-25T13:18:01.879+0000 7f00b96bb700 -1 mds.l *** got signal Terminated *** 2024-04-25T13:18:05.029 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:04 smithi178 podman[71223]: 2024-04-25 13:18:04.747544381 +0000 UTC m=+2.975756699 container died 498a421645133b4bea3378a32db40d5349b7c862104dfe911061c1f5b0811317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:18:05.029 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:05 smithi178 podman[71223]: 2024-04-25 13:18:05.029439143 +0000 UTC m=+3.257651466 container cleanup 498a421645133b4bea3378a32db40d5349b7c862104dfe911061c1f5b0811317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T13:18:05.346 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:05 smithi178 bash[71223]: ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l 2024-04-25T13:18:05.347 INFO:journalctl@ceph.mds.l.smithi178.stdout:Apr 25 13:18:05 smithi178 podman[71238]: 2024-04-25 13:18:05.247245213 +0000 UTC m=+0.500780289 container remove 498a421645133b4bea3378a32db40d5349b7c862104dfe911061c1f5b0811317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d3627c2a-0302-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:18:05.363 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3627c2a-0302-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T13:18:05.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:18:05.393 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T13:18:05.393 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:18:05.593 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:18:08.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:18:08.434 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:18:10.806 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:18:10.995 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:18:13.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:18:13.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:18:13.378 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:18:13.404 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T13:18:13.406 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi038/crash 2024-04-25T13:18:13.407 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:18:13.465 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:18:13.465 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:18:13.467 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi104/crash 2024-04-25T13:18:13.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:18:13.520 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:18:13.520 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:18:13.522 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi178/crash 2024-04-25T13:18:13.523 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:18:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:18:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:18:13.559 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T13:18:13.559 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T13:18:13.597 INFO:tasks.cephadm:Compressing logs... 2024-04-25T13:18:13.597 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:18:13.640 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:18:13.643 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:18:13.665 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:18:13.665 INFO:teuthology.orchestra.run.smithi178.stderr:: No such file or directory 2024-04-25T13:18:13.666 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T13:18:13.673 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T13:18:13.674 INFO:teuthology.orchestra.run.smithi178.stderr: 87.7% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:18:13.674 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:18:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:18:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:: No such file or directory 2024-04-25T13:18:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:18:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.g.log/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log: 2024-04-25T13:18:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr: 90.3% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log.gz/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.6.log: 2024-04-25T13:18:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T13:18:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:18:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:18:13.684 INFO:teuthology.orchestra.run.smithi038.stderr: 94.6% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T13:18:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T13:18:13.685 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T13:18:13.685 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:18:13.685 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T13:18:13.685 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T13:18:13.685 INFO:teuthology.orchestra.run.smithi178.stderr: 83.7% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T13:18:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.h.log 87.7% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.10.log: 90.5% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr: 91.9% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T13:18:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:18:13.689 INFO:teuthology.orchestra.run.smithi104.stderr: 83.7% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:18:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:18:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T13:18:13.692 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:18:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T13:18:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:18:13.695 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T13:18:13.705 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T13:18:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T13:18:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:18:13.713 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:18:13.714 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:18:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.l.log: 93.2% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T13:18:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:18:13.728 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:18:13.729 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:18:13.731 INFO:teuthology.orchestra.run.smithi178.stderr: 91.0% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:18:13.732 INFO:teuthology.orchestra.run.smithi104.stderr: 93.0% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T13:18:13.736 INFO:teuthology.orchestra.run.smithi038.stderr: 94.5% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T13:18:13.737 INFO:teuthology.orchestra.run.smithi178.stderr: 94.1% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T13:18:13.743 INFO:teuthology.orchestra.run.smithi038.stderr: 93.2% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T13:18:13.746 INFO:teuthology.orchestra.run.smithi178.stderr: 92.9% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T13:18:13.748 INFO:teuthology.orchestra.run.smithi104.stderr: 94.6% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T13:18:13.778 INFO:teuthology.orchestra.run.smithi178.stderr: 94.4% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T13:18:13.791 INFO:teuthology.orchestra.run.smithi104.stderr: 94.4% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T13:18:13.889 INFO:teuthology.orchestra.run.smithi104.stderr: 93.4% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T13:18:13.932 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T13:18:13.981 INFO:teuthology.orchestra.run.smithi104.stderr: 92.0% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T13:18:14.033 INFO:teuthology.orchestra.run.smithi038.stderr: 90.2% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T13:18:14.209 INFO:teuthology.orchestra.run.smithi178.stderr: 92.9% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T13:18:14.367 INFO:teuthology.orchestra.run.smithi104.stderr: 92.2% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T13:18:14.374 INFO:teuthology.orchestra.run.smithi178.stderr: 92.1% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T13:18:14.468 INFO:teuthology.orchestra.run.smithi038.stderr: 90.5% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T13:18:14.517 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T13:18:14.631 INFO:teuthology.orchestra.run.smithi104.stderr: 92.3% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T13:18:14.688 INFO:teuthology.orchestra.run.smithi038.stderr: 93.2% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T13:18:14.720 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T13:18:14.751 INFO:teuthology.orchestra.run.smithi104.stderr: 93.0% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T13:18:14.833 INFO:teuthology.orchestra.run.smithi104.stderr: 93.0% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T13:18:14.835 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-04-25T13:18:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m1.179s 2024-04-25T13:18:14.836 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m4.461s 2024-04-25T13:18:14.836 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m0.174s 2024-04-25T13:18:14.973 INFO:teuthology.orchestra.run.smithi038.stderr: 92.9% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T13:18:15.054 INFO:teuthology.orchestra.run.smithi178.stderr: 92.9% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T13:18:15.056 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-04-25T13:18:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m1.400s 2024-04-25T13:18:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m4.039s 2024-04-25T13:18:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.162s 2024-04-25T13:18:15.163 INFO:teuthology.orchestra.run.smithi038.stderr: 92.7% -- replaced with /var/log/ceph/d3627c2a-0302-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T13:18:15.167 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T13:18:15.167 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m1.510s 2024-04-25T13:18:15.167 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m5.791s 2024-04-25T13:18:15.167 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.204s 2024-04-25T13:18:15.167 INFO:tasks.cephadm:Archiving logs... 2024-04-25T13:18:15.168 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi038/log 2024-04-25T13:18:15.169 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:18:15.986 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi104/log 2024-04-25T13:18:15.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:18:16.646 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi178/log 2024-04-25T13:18:16.647 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:18:17.251 INFO:tasks.cephadm:Removing cluster... 2024-04-25T13:18:17.251 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --force 2024-04-25T13:18:17.441 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:18:17.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --force 2024-04-25T13:18:17.901 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:18:18.150 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d3627c2a-0302-11ef-bc93-c7b262605968 --force 2024-04-25T13:18:18.340 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: d3627c2a-0302-11ef-bc93-c7b262605968 2024-04-25T13:18:18.594 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T13:18:18.594 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:18:18.613 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:18:18.629 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:18:18.646 INFO:tasks.cephadm:Teardown complete 2024-04-25T13:18:18.646 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T13:18:18.661 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T13:18:18.661 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:18:18.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:18:18.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:18:18.781 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T13:18:18.792 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T13:18:18.792 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:18:18.795 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:18:18.797 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:18:18.808 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:18:18.809 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:18:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:18:18.813 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:18:18.813 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-04-25T13:18:18.814 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +904us[ +900us] +/- 49ms 2024-04-25T13:18:18.814 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1531us[-1536us] +/- 57ms 2024-04-25T13:18:18.814 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +512us[ +508us] +/- 65ms 2024-04-25T13:18:18.814 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:18:18.815 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +890us[ +896us] +/- 49ms 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 -1554us[-1548us] +/- 57ms 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +488us[ +494us] +/- 65ms 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:18:18.816 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-04-25T13:18:18.817 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +892us[ +890us] +/- 49ms 2024-04-25T13:18:18.817 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -1538us[-1538us] +/- 57ms 2024-04-25T13:18:18.817 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +508us[ +506us] +/- 65ms 2024-04-25T13:18:18.817 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:18:18.817 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T13:18:18.830 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T13:18:18.831 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T13:18:18.859 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:18:18.938 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:18:19.001 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:18:19.064 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:18:19.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:18:19.095 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:18:19.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:18:19.121 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:18:19.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:18:19.148 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T13:18:19.162 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T13:18:19.199 INFO:teuthology.task.internal:Duration was 1817.333474 seconds 2024-04-25T13:18:19.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T13:18:19.232 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T13:18:19.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:18:19.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:18:19.238 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:18:19.271 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:18:19.273 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:18:19.275 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:18:19.614 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T13:18:19.615 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T13:18:19.615 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:18:19.679 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-04-25T13:18:19.679 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:18:19.703 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T13:18:19.703 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:18:19.730 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T13:18:19.730 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:18:19.733 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:18:19.745 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:18:19.795 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T13:18:19.795 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:18:19.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:18:19.811 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:18:20.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T13:18:20.216 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T13:18:20.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:18:20.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:18:20.263 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:18:20.287 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T13:18:20.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:18:20.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:18:20.306 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:18:20.321 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-04-25T13:18:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-04-25T13:18:20.350 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-04-25T13:18:20.367 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:18:20.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:18:20.389 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:18:20.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:18:20.403 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:18:20.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:18:20.421 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T13:18:20.436 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T13:18:20.437 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi038 2024-04-25T13:18:20.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:18:20.478 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi104 2024-04-25T13:18:20.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:18:20.513 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453/remote/smithi178 2024-04-25T13:18:20.513 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:18:20.548 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T13:18:20.549 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:18:20.551 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:18:20.553 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:18:20.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T13:18:20.615 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T13:18:20.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T13:18:20.640 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T13:18:20.640 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:18:20.643 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:18:20.645 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:18:20.655 INFO:teuthology.orchestra.run.smithi038.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:18 /home/ubuntu/cephtest 2024-04-25T13:18:20.657 INFO:teuthology.orchestra.run.smithi104.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:18 /home/ubuntu/cephtest 2024-04-25T13:18:20.659 INFO:teuthology.orchestra.run.smithi178.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:18 /home/ubuntu/cephtest 2024-04-25T13:18:20.660 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T13:18:20.674 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T13:18:20.740 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T13:18:20.766 DEBUG:teuthology.nuke:shortname: smithi038 2024-04-25T13:18:20.767 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:18:20.793 DEBUG:teuthology.nuke:shortname: smithi104 2024-04-25T13:18:20.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:18:20.799 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:42:10.113651', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7sPohrFa/ycNsFwzLrFpz+tFbDl/kKeJBi5ZXHCpl8jNxfniMPBXsVBzQP5RTkXPPkSpwM+KCQOPp5JB3mGGY='} 2024-04-25T13:18:20.818 DEBUG:teuthology.nuke:shortname: smithi178 2024-04-25T13:18:20.819 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:18:20.823 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:42:10.112037', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJIK5jD5ANCb/OquUGo45AIch0aciP7DNLp7PDAlntG2ktHGtcIWL0sqo8MAFIvr1iyUOOxIAu25mqnUgfReBSw='} 2024-04-25T13:18:20.852 INFO:teuthology.orchestra.console.smithi038:Power off 2024-04-25T13:18:20.853 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:18:20.869 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673453', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:42:10.110136', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxexWztidPuKK87DQUoYQxOHZKXtWTDMt67g8lV0g6JLZVF9NUfIrNeqpayPj/Otx9W8DsVWqHLXOqWmaZmWvQ='} 2024-04-25T13:18:20.887 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-04-25T13:18:20.887 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:20.905 INFO:teuthology.orchestra.console.smithi104:Power off 2024-04-25T13:18:20.905 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:18:20.918 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-25T13:18:20.930 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off 2024-04-25T13:18:20.930 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:20.946 INFO:teuthology.orchestra.console.smithi178:Power off 2024-04-25T13:18:20.947 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:18:20.956 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-04-25T13:18:20.971 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2024-04-25T13:18:20.971 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:20.997 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-04-25T13:18:24.919 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:24.959 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:24.998 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:25.029 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-25T13:18:25.069 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-04-25T13:18:25.109 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-04-25T13:18:29.034 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:29.070 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:29.110 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:29.145 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-25T13:18:29.179 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-04-25T13:18:29.220 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2024-04-25T13:18:29.280 INFO:teuthology.orchestra.console.smithi104:Power off completed 2024-04-25T13:18:29.321 INFO:teuthology.orchestra.console.smithi178:Power off completed 2024-04-25T13:18:33.148 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:18:33.262 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-04-25T13:18:33.363 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-04-25T13:18:33.468 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} duration: 1817.3334741592407 failure_reason: '''NoneType'' object has no attribute ''get''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d12c9abd447b4b4ba48d869e564148b3 status: fail success: false 2024-04-25T13:18:33.468 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:18:33.545 INFO:teuthology.run:FAIL